-- Logs begin at Mon 2022-08-22 16:08:45 UTC, end at Mon 2022-08-22 17:56:17 UTC. -- Aug 22 17:04:01.369208 dv-cinder-853941-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 22 17:04:01.376550 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 22 17:04:01 2022] *** Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: nodename: dv-cinder-853941-2-iscsi Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: machine: x86_64 Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: clock source: unix Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: pcre jit disabled Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: detected number of CPU cores: 4 Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: current working directory: / Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: detected binary path: /usr/bin/uwsgi-core Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: your processes number limit is 47680 Aug 22 17:04:01.379667 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: your memory page size is 4096 bytes Aug 22 17:04:01.380353 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: detected max file descriptor number: 2048 Aug 22 17:04:01.380353 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: lock engine: pthread robust mutexes Aug 22 17:04:01.380353 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: thunder lock: enabled Aug 22 17:04:01.380353 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 22 17:04:01.380353 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 22 17:04:01.419643 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: Python main interpreter initialized at 0x56506b49f9c0 Aug 22 17:04:01.419643 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: python threads support enabled Aug 22 17:04:01.419643 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: your server socket listen backlog is limited to 100 connections Aug 22 17:04:01.419643 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: your mercy for graceful operations on workers is 90 seconds Aug 22 17:04:01.420036 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: mapped 403077 bytes (393 KB) for 2 cores Aug 22 17:04:01.420110 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: *** Operational MODE: preforking *** Aug 22 17:04:01.420352 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: *** uWSGI is running in multiple interpreter mode *** Aug 22 17:04:01.420352 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: spawned uWSGI master process (pid: 82131) Aug 22 17:04:01.420566 dv-cinder-853941-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 22 17:04:01.423076 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: spawned uWSGI worker 1 (pid: 82145, cores: 1) Aug 22 17:04:01.423076 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: spawned uWSGI worker 2 (pid: 82146, cores: 1) Aug 22 17:04:01.423076 dv-cinder-853941-2-iscsi devstack@keystone.service[82131]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 22 17:04:05.797011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 17:04:05.797011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: from cryptography.utils import int_from_bytes Aug 22 17:04:05.797810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 17:04:05.797810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: from cryptography.utils import int_from_bytes Aug 22 17:04:05.798583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 17:04:05.798583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: from cryptography.utils import int_from_bytes Aug 22 17:04:05.799059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 17:04:05.799059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: from cryptography.utils import int_from_bytes Aug 22 17:04:05.849526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 22 17:04:05.849731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 22 17:04:05.849893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] command line args: [] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 22 17:04:05.850170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 22 17:04:05.850399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ================================================================================ {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 22 17:04:05.850513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 22 17:04:05.850604 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 22 17:04:05.850796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] admin_token = **** {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.850995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] command line args: [] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 22 17:04:05.851080 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] config_dir = [] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.851226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.851305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 22 17:04:05.851419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] config_source = [] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.851634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ================================================================================ {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 22 17:04:05.851727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] debug = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.851805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] admin_token = **** {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.851884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.852066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] config_dir = [] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.852131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.852194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.852420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] config_source = [] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.852542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.852621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] debug = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.852691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.852757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.852861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.852977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.853057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] list_limit = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.853124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.853369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] log_config_append = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.853463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.853527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.853621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] log_dir = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.853735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.853841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] log_file = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.853947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] list_limit = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] log_config_append = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854299 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] log_dir = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] log_file = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.854993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.855055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.855254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.855372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.855447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.855637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.855707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.855781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.855846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.856006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.856093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.856186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.856305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.856418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.856524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.856722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.856825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.856896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.856960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.857157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.857262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.857410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.857670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.857751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.857936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] publish_errors = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.858020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.858082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.858276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.858457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.858590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.858695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] publish_errors = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.858761 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.858929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.859007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.859160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.859236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] standard_threads = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.859422 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.859508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.859622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.859702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.859820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.859942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.860059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] use_journal = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.860169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] standard_threads = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.860271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] use_json = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.860393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.860475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] use_stderr = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.860475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] use_syslog = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.860651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.860651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.860885 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.860978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] use_journal = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.861095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.861221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] use_json = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.861221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] use_stderr = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.861366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.861447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] use_syslog = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.861523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.861600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 17:04:05.861728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.861804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.861931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.862007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.862077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.862225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.862308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.862469 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.862582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.862745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.862829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.862990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.863074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.863295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.863385 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.863533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.863618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.863784 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.863861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.864020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.864108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.864269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.864354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.864508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.864591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.864746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.864834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.864988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.865077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.865237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.865314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.865482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.865568 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.865724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.866343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.866530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.866620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.866807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.866888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.867007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.867082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.867236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.867321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.867494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.867574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.867746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.867821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.867980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.868065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.868219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.868299 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.868461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.868539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.868694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.868772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.868928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.869006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.869161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.869238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.869409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.869488 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.869659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.869737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.869892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.869975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.870127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.870239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.870344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.870440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.870603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.870751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.870862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.870960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.871066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.871169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.871272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.871391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.871495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.871574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.871728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.872010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.872096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.872216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.872357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.872435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.872563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.872666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.872794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.872895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.872972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.873130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.873219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.873359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.873475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.873584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.873720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.873820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.873910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.874058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.874136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.874293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.874414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.874563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.874694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.874774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.874898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.875025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.875148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.875224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.875298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.875474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.875555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.875708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.875788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.875945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.876034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.876176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.876254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.876425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.876506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.876663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.876750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.876894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.876978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.877126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.877217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.877363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.877449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.877616 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.877700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.877849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.877933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.878094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.878251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.878333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.878405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.878593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.878676 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.878834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.878929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.879074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.879153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.879322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.879407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.879562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.879654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.879798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.879884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.880038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.880119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.880278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.880357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.880521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.880612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.880760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.880838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.880998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.881075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.881237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.881316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.882045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.882190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.882279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.882353 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.882527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.882637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.882770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.882877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.883039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.883196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.883285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.883442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.883521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.883684 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.883763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.883926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.884012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.884161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.884240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.884395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.884473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.884627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.884715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.884868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.884953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.885099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.885233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.885317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.885398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.885558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.885637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.885792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.885872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.886029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.886117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.886263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.886341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.886550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.886633 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.886775 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.886867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.887008 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] auth.external = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.887086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.887240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.887322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.887511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.887597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.887745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] auth.external = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.887824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.887977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.888065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] auth.password = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.888205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] auth.token = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.888284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.888446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.888537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.888682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.888761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] auth.password = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.888928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.889013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] auth.token = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.889160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.889251 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.889396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.889474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.889646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.889725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.889876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.889966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.890115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] credential.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.890194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.890364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.890443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.890622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.890715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.890864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] credential.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.890943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.891108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.891186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.891359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.891445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.891595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.891674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.891839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.891916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.892089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.892180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.892326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.892404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.892578 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.892656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.892834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.892996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.893073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.893145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.893320 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.893399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.893580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.893666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.893838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.893917 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.894075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] federation.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.894153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.894314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.894401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.894586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.894669 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.894836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.894915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] federation.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.895069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.895160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.895317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.895397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.895557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.895636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.895790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.895870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.896041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.896121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.896283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.896361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.896521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.896600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.896770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.896849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.897032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.897120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.897377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.897523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.897709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.898006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.898086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.898339 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.898425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.898624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.898708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.898864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.898944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.899112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.899199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.899358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.899437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.899614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.899692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.899849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.899936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.900082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.900161 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.900318 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.900396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.900567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.900654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.900805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.900884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.901044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.901130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.901298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.901376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.901548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.901617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.901798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.901865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.902051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.902128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.902290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.902363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.902547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.902622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.902792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.902874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.903028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.903097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.903268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.903349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.903512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.903600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.903755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.903834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.903988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.904067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.904226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.904312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.904470 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.904548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.904714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.904794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.904947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.905027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.905189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.905267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.905461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.905555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.905670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.905752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.905910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.905991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.906181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.906275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.906380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.906462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.906645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.906736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.906888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.906968 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.907119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.907212 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.907368 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.907471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.907600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.907679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.907846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.907927 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.908082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.908173 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.908309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.908391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.908558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.908639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.908806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.908885 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.909039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.909118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.909271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.909351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.909507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.909585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.909738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.909818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.909979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.910060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.910215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.910383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.910463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.910651 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.910745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.910904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.910983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.911154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.911331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.911410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.911573 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.911654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.911825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.911904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.912073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.912151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.912313 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.912394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.912553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.912714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.912934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.913014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.913182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.913400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.913480 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.913663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.913882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.913961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.914140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.914219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.914391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.914469 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.914667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.914757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.914922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.915003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.915173 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.915255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.915409 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.915489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.915655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.915735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.915894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.915972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.916131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.916209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.916371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.916450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.916611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.916689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.916847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.916926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.917097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.917178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.917326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.917404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.917579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.917658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.917810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.917890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.918043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.918121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.918282 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.918370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] resource.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.919950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.920020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] resource.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.920133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.920210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.920379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.920480 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.920615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.920711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.920873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.921012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.921108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.921189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.921348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.921436 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.921589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.921679 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.921830 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.921916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.922066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] role.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.922152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.922302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] role.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.922386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] role.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.922568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.922663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] role.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.922838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.922929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.923070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.923157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.923319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.923403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.923564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.923649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.923836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.923921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.924108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.924195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.924375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.924461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.924628 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.924713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.924878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.924972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.925143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.925229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.925413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.925497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.925930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.926027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.926181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.926267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.926429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.926646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.926898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.926984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.927142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.927229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.927416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.927496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.927695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.927787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.927982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.928069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.928793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.928970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.929052 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.929290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.929386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.929558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.929680 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.929800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.929893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.930039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.930127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.930527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.930678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.930678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.930807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.930889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.930963 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.931084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.931164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.931377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.931462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.931619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.931708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.931859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.931947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.932094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.932180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.932332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] token.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.932414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] token.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.932574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.932665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.932824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.932905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.933062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.933148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.933318 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.933404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.933571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.933657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.933813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.933895 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.934061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.934142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.934305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.934384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.934580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.934666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.934822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.934949 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.935060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.935218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.935314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.935510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.935664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.935765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.935844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.935996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.936084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.936242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.936323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.936530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.936614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.936795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.connection = **** {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.936889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.937043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.937124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.connection = **** {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.937292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.937424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.937544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.937701 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.937791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.937871 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.938048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.938134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.938246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.938386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.938471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.938672 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.938766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.938955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.939077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.939189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.939293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.939421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.939562 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.939692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.939775 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.939895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.939977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.940132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.940225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.940381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.940463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.940613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.940707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.940882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.940971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.941151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.941289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.941375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 22 17:04:05.941448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.941649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 17:04:05.941840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 22 17:04:05.972889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:05.973083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:05.973221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:05.973343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:05.973442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:05.973530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:05.982615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:05.982826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.011718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.011940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.012886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.013119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.067055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.068094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.117618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.117875 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.118183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.118425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.280169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.280395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.280661 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.280889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.281140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.281321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.286938 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.287226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.287503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.288645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.288645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.289194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.307578 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.307938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.308090 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.308228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.308363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.308528 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.308699 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.308894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.309088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.309219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.309350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.309496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.309631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.309631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.320495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.320814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.333673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.333929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.334075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.334212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.334346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.334455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.335172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.335305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.335432 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.335565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.335710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.335855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.335983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.336122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 17:04:06.353224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.353502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 17:04:06.353713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.353713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 17:04:06.353972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.353972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 17:04:06.354209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.354209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 17:04:06.354467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.354619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.354757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.354887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.355044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.355298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.355431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.355555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.364811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.365091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.365230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.365372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.365508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.365675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.365840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.365970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.373039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.373191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.375995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.376139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.376265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.376401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.376543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.376681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.376815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.376939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.377100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.377229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.377352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.377480 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.377615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.377749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.377890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.378020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.378267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.378394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.378558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.378698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.378848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.378975 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.402206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.402700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.403720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.404012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.404180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.404473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.408754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.409038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.409223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.409350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.409517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.409809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.416406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.416693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.416882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.417176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.420849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.421141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.421341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.421492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.421628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.421759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.421895 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.422024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.422189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.422332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.422467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.422624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.422761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.422897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.423032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.423295 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.440658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.440959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.441246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.441479 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.444979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.445471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.448241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.448543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.448762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.449070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.452783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.453331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.455974 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.456540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.459282 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.460099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.460390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.460828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.460981 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.461129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.461264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.461397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.461538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.461683 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.461683 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.461888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.477909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.478194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.478472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.478637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.478770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.478900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.479071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.479344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.485707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.486337 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.487262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.487546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.487913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.488197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.490291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.490650 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.491428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.491568 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.491725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.491814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.492012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.492106 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.492339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.492622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.503966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.504105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.504258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.504396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.504540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.504684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.504823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.504973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.505112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.505264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.505412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.505578 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.505719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.505856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.521435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.521682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.525201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.525535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.528137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.528510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.529588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.529989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.531142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.531387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.531521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.531690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.531822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.532080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.538591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.538810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.540293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.540495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.543245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.543446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.543629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.543803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.543979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.544153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.544320 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.544472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.552403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.552562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.552698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.552825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.552964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.553154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.560508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.560754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 17:04:06.560890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.561051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.561166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.561335 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.561422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.561607 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.561692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.561877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.561988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.562156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.562243 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.562427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.562526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.562725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.562809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.563040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.563138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.563297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.563378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.563565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.563655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.563827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.563916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.564099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.564326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.564417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 17:04:06.599369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56506b49f9c0 pid: 82145 (default app) Aug 22 17:04:06.600314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56506b49f9c0 pid: 82146 (default app) Aug 22 17:04:06.603082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf362798-edd1-4d9d-b2b8-1ead089cd2dc None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:06.603382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf362798-edd1-4d9d-b2b8-1ead089cd2dc None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:06.603744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf362798-edd1-4d9d-b2b8-1ead089cd2dc None None] PATH_INFO: `/v3` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:06.607865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Mon Aug 22 17:04:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:14.657356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fa578-785f-4821-8e5f-9a40d0677f8d None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:14.657918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fa578-785f-4821-8e5f-9a40d0677f8d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:14.657918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fa578-785f-4821-8e5f-9a40d0677f8d None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:14.662098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:04:14.670261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:14.670779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:14.670887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:14.743624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: INFO dbcounter [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] Registered counter for database keystone Aug 22 17:04:14.765769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG oslo_db.sqlalchemy.engines [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82145) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 22 17:04:14.772257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writer thread running {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 22 17:04:14.959714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG passlib.handlers.bcrypt [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=82145) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 22 17:04:14.960227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG passlib.handlers.bcrypt [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82145) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 22 17:04:15.016966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:15.018616 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:15.156266 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 22 17:04:15.622658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fa578-785f-4821-8e5f-9a40d0677f8d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:15.707332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: INFO dbcounter [None req-a85fa578-785f-4821-8e5f-9a40d0677f8d None None] Registered counter for database keystone Aug 22 17:04:15.727888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG oslo_db.sqlalchemy.engines [None req-a85fa578-785f-4821-8e5f-9a40d0677f8d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82146) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 22 17:04:15.734613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writer thread running {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 22 17:04:15.864137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fa578-785f-4821-8e5f-9a40d0677f8d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:15.875676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616e2cf-c4ec-4d7e-9ba7-5cc87a84184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:15.876304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616e2cf-c4ec-4d7e-9ba7-5cc87a84184c admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:15.876586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616e2cf-c4ec-4d7e-9ba7-5cc87a84184c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:15.876861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616e2cf-c4ec-4d7e-9ba7-5cc87a84184c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:15.886373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616e2cf-c4ec-4d7e-9ba7-5cc87a84184c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:15.938789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6616e2cf-c4ec-4d7e-9ba7-5cc87a84184c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 22 17:04:15.939133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616e2cf-c4ec-4d7e-9ba7-5cc87a84184c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:15.950367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Aug 22 17:04:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 17:04:15.956673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:15.974492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23db302-da14-427d-abdd-9aa1404dd9ba None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:15.979329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dd2bf-d675-409b-aabd-787e331101f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:15.982804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dd2bf-d675-409b-aabd-787e331101f2 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:15.982804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dd2bf-d675-409b-aabd-787e331101f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:15.982804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dd2bf-d675-409b-aabd-787e331101f2 admin admin] PATH_INFO: `/v3/projects/14e122ee73be44deb3dfe27923205d92` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:15.982804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dd2bf-d675-409b-aabd-787e331101f2 admin admin] RBAC: Authorizing `identity:get_project(project_id=14e122ee73be44deb3dfe27923205d92)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:16.036393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: WARNING keystone.common.rbac_enforcer.enforcer [None req-028dd2bf-d675-409b-aabd-787e331101f2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 22 17:04:16.036734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dd2bf-d675-409b-aabd-787e331101f2 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:16.041490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Aug 22 17:04:15 2022] GET /identity/v3/projects/14e122ee73be44deb3dfe27923205d92 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:17.988179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774524d3-9306-4d6d-b625-a6afa62b129f None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:17.988699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774524d3-9306-4d6d-b625-a6afa62b129f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:17.988857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774524d3-9306-4d6d-b625-a6afa62b129f None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:17.992976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:17.998850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013600d-6d7a-4b39-9b98-bd8d6b5569c1 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:17.999282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013600d-6d7a-4b39-9b98-bd8d6b5569c1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:17.999464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013600d-6d7a-4b39-9b98-bd8d6b5569c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:18.020455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a013600d-6d7a-4b39-9b98-bd8d6b5569c1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:18.021976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a013600d-6d7a-4b39-9b98-bd8d6b5569c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:18.030431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 22 17:04:18.306295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774524d3-9306-4d6d-b625-a6afa62b129f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:18.315124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774524d3-9306-4d6d-b625-a6afa62b129f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:18.319268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be98bf6-6edb-479e-b21a-75d1e0bea367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:18.320132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be98bf6-6edb-479e-b21a-75d1e0bea367 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:18.320597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be98bf6-6edb-479e-b21a-75d1e0bea367 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:18.320876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be98bf6-6edb-479e-b21a-75d1e0bea367 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:18.328944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be98bf6-6edb-479e-b21a-75d1e0bea367 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:18.330766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be98bf6-6edb-479e-b21a-75d1e0bea367 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:18.340865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Aug 22 17:04:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:04:18.346842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013600d-6d7a-4b39-9b98-bd8d6b5569c1 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:18.357046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013600d-6d7a-4b39-9b98-bd8d6b5569c1 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:18.361446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f11980-d7f4-4a22-9b73-0991b58a31ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:18.362080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11980-d7f4-4a22-9b73-0991b58a31ea admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:18.362352 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11980-d7f4-4a22-9b73-0991b58a31ea admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:18.362649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11980-d7f4-4a22-9b73-0991b58a31ea admin admin] PATH_INFO: `/v3/users/b8814a79ac534e6e8b81d70404045912` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:18.363960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f11980-d7f4-4a22-9b73-0991b58a31ea admin admin] RBAC: Authorizing `identity:get_user(user_id=b8814a79ac534e6e8b81d70404045912)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:18.365786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f11980-d7f4-4a22-9b73-0991b58a31ea admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:18.367778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Mon Aug 22 17:04:18 2022] GET /identity/v3/users/b8814a79ac534e6e8b81d70404045912 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:20.374430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd175c3-6420-4404-ab03-b97bdeeff2d1 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:20.375000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd175c3-6420-4404-ab03-b97bdeeff2d1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:20.375000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd175c3-6420-4404-ab03-b97bdeeff2d1 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:20.376334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:04:20.381897 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cd203-02a3-407a-87c1-c620c428e803 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:20.382611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cd203-02a3-407a-87c1-c620c428e803 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:20.382611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cd203-02a3-407a-87c1-c620c428e803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:20.390433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a9067-3289-488e-b281-d6c38567dd3a None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:20.390737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a9067-3289-488e-b281-d6c38567dd3a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:20.391010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a9067-3289-488e-b281-d6c38567dd3a None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:20.392320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:04:20.397476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89f51c-0ffd-4db2-ba34-53bde21ad861 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:20.397765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89f51c-0ffd-4db2-ba34-53bde21ad861 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:20.398038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89f51c-0ffd-4db2-ba34-53bde21ad861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:20.401638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-7a7cd203-02a3-407a-87c1-c620c428e803 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:20.402943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7a7cd203-02a3-407a-87c1-c620c428e803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:20.443365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG passlib.handlers.bcrypt [None req-5f89f51c-0ffd-4db2-ba34-53bde21ad861 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=82146) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 22 17:04:20.443793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG passlib.handlers.bcrypt [None req-5f89f51c-0ffd-4db2-ba34-53bde21ad861 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82146) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 22 17:04:20.489160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-5f89f51c-0ffd-4db2-ba34-53bde21ad861 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:20.490823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5f89f51c-0ffd-4db2-ba34-53bde21ad861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:20.491175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:20.592314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:04:20.770761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cd203-02a3-407a-87c1-c620c428e803 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:20.795678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:20.857864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89f51c-0ffd-4db2-ba34-53bde21ad861 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:20.884961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cd203-02a3-407a-87c1-c620c428e803 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:20.921244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38953cad-50a9-4f42-9fe3-fee822804ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:20.921889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38953cad-50a9-4f42-9fe3-fee822804ccf None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:20.922170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38953cad-50a9-4f42-9fe3-fee822804ccf None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:20.922446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38953cad-50a9-4f42-9fe3-fee822804ccf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:20.935730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38953cad-50a9-4f42-9fe3-fee822804ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-38953cad-50a9-4f42-9fe3-fee822804ccf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:20.944574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:20.945758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:20.946687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:04:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:20.954070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38953cad-50a9-4f42-9fe3-fee822804ccf None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:20.971552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89f51c-0ffd-4db2-ba34-53bde21ad861 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:20.973928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:21.010999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe5cf9-25bf-4203-8ab3-3e3a8c0f35ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:21.011994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe5cf9-25bf-4203-8ab3-3e3a8c0f35ad None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:21.012351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe5cf9-25bf-4203-8ab3-3e3a8c0f35ad None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:21.012634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe5cf9-25bf-4203-8ab3-3e3a8c0f35ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:21.039298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:21.058931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38953cad-50a9-4f42-9fe3-fee822804ccf None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:21.094822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe5cf9-25bf-4203-8ab3-3e3a8c0f35ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:21.096352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe5cf9-25bf-4203-8ab3-3e3a8c0f35ad None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:21.096608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a68dc-2dcb-4051-8c2f-799352a517d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:21.097219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a68dc-2dcb-4051-8c2f-799352a517d8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:21.097495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a68dc-2dcb-4051-8c2f-799352a517d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:21.097765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a68dc-2dcb-4051-8c2f-799352a517d8 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:21.098437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a68dc-2dcb-4051-8c2f-799352a517d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:21.098764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a68dc-2dcb-4051-8c2f-799352a517d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:21.100151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a68dc-2dcb-4051-8c2f-799352a517d8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:21.114023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:21.120629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a68dc-2dcb-4051-8c2f-799352a517d8 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:21.205044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 17:04:21.211251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe5cf9-25bf-4203-8ab3-3e3a8c0f35ad None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:21.214113 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:21.238311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a68dc-2dcb-4051-8c2f-799352a517d8 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:21.275261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0dbbb-71f0-49ea-8350-39d76703f1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:21.275904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0dbbb-71f0-49ea-8350-39d76703f1dd None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:21.276304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0dbbb-71f0-49ea-8350-39d76703f1dd None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:21.276584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0dbbb-71f0-49ea-8350-39d76703f1dd None admin] PATH_INFO: `/v3/users/b8814a79ac534e6e8b81d70404045912` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:21.277721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0dbbb-71f0-49ea-8350-39d76703f1dd None admin] RBAC: Authorizing `identity:get_user(user_id=b8814a79ac534e6e8b81d70404045912)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:21.279223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0dbbb-71f0-49ea-8350-39d76703f1dd None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:21.280808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:21 2022] GET /identity/v3/users/b8814a79ac534e6e8b81d70404045912 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:21.286531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0dbbb-71f0-49ea-8350-39d76703f1dd None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:21.302104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:21.318631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe5cf9-25bf-4203-8ab3-3e3a8c0f35ad None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:21.359200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e5635-f0b5-4ce2-9a99-6f769f99a412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:21.360063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e5635-f0b5-4ce2-9a99-6f769f99a412 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:21.360342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e5635-f0b5-4ce2-9a99-6f769f99a412 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:21.360609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e5635-f0b5-4ce2-9a99-6f769f99a412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:21.379900 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e5635-f0b5-4ce2-9a99-6f769f99a412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:21.381383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e5635-f0b5-4ce2-9a99-6f769f99a412 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:21.396873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0dbbb-71f0-49ea-8350-39d76703f1dd None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-636e5635-f0b5-4ce2-9a99-6f769f99a412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:04:21.400196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:21.408016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:04:21.408876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:04:21.408876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:04:21.408876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:21.408876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:04:21.408876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e5635-f0b5-4ce2-9a99-6f769f99a412 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:21.436408 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0c8a2-a428-422f-9e64-0845970cf6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:21.437124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0c8a2-a428-422f-9e64-0845970cf6b2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:21.437329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0c8a2-a428-422f-9e64-0845970cf6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:21.437645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0c8a2-a428-422f-9e64-0845970cf6b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:21.438824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0c8a2-a428-422f-9e64-0845970cf6b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:21.440420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0c8a2-a428-422f-9e64-0845970cf6b2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:21.441935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:21.448681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0c8a2-a428-422f-9e64-0845970cf6b2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:21.466607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:21.513039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e5635-f0b5-4ce2-9a99-6f769f99a412 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:21.532827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:21.547698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43664f1a-24dc-443b-ac13-f06b455bf4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:21.548315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43664f1a-24dc-443b-ac13-f06b455bf4db None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:21.548599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43664f1a-24dc-443b-ac13-f06b455bf4db None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:21.548873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43664f1a-24dc-443b-ac13-f06b455bf4db None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:21.549521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43664f1a-24dc-443b-ac13-f06b455bf4db None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:21.550076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43664f1a-24dc-443b-ac13-f06b455bf4db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:21.551528 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43664f1a-24dc-443b-ac13-f06b455bf4db None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:21.552194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0c8a2-a428-422f-9e64-0845970cf6b2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:21.571966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:21.590976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3655944-c6ea-4057-ae52-aa6504f610a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:21.591618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3655944-c6ea-4057-ae52-aa6504f610a8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:21.591902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3655944-c6ea-4057-ae52-aa6504f610a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:21.592179 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3655944-c6ea-4057-ae52-aa6504f610a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:21.592939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3655944-c6ea-4057-ae52-aa6504f610a8 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.domain.id=default, user.id=b8814a79ac534e6e8b81d70404045912) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:21.593252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3655944-c6ea-4057-ae52-aa6504f610a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:21.594776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3655944-c6ea-4057-ae52-aa6504f610a8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:21.602834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Aug 22 17:04:21 2022] GET /identity/v3/role_assignments?user.id=b8814a79ac534e6e8b81d70404045912&scope.domain.id=default&role.id=4b10782d99b74f5bb453b82286012e0c => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:23.678856 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81a9f-78f7-4375-87c5-de43d0bd90e8 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:23.679541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81a9f-78f7-4375-87c5-de43d0bd90e8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:23.679541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81a9f-78f7-4375-87c5-de43d0bd90e8 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:23.681105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:23.686634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff9dc0-260f-4743-b8ac-e1d708449b96 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:23.687532 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff9dc0-260f-4743-b8ac-e1d708449b96 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:23.688003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff9dc0-260f-4743-b8ac-e1d708449b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:23.711762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b5ff9dc0-260f-4743-b8ac-e1d708449b96 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:23.713039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b5ff9dc0-260f-4743-b8ac-e1d708449b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:23.805815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:24.051381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7365c-4dc4-4b6d-96b9-75f053de4d86 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:24.052053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7365c-4dc4-4b6d-96b9-75f053de4d86 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:24.052548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7365c-4dc4-4b6d-96b9-75f053de4d86 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:24.054299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 12/24] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:24.060265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b00a56-07ec-4c01-bca9-794832461c9e None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:24.060553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b00a56-07ec-4c01-bca9-794832461c9e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:24.060828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b00a56-07ec-4c01-bca9-794832461c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:24.090004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-09b00a56-07ec-4c01-bca9-794832461c9e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:24.091265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-09b00a56-07ec-4c01-bca9-794832461c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:24.118391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f585ffc-8dd2-46dd-95eb-9c6a477060e5 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:24.119033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f585ffc-8dd2-46dd-95eb-9c6a477060e5 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:24.119522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f585ffc-8dd2-46dd-95eb-9c6a477060e5 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:24.121360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 13/25] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:24.126770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47305154-a23a-4049-8805-b673ada29b01 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:24.127307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47305154-a23a-4049-8805-b673ada29b01 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:24.127772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47305154-a23a-4049-8805-b673ada29b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:24.167583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-47305154-a23a-4049-8805-b673ada29b01 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:24.169135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-47305154-a23a-4049-8805-b673ada29b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:24.171555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 13/26] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:04:24.278645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 14/27] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:24.362673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b00a56-07ec-4c01-bca9-794832461c9e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:24.367351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:24.527130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b00a56-07ec-4c01-bca9-794832461c9e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:24.580014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f1af1-743e-4dfe-b416-8774e21becc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:24.580707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f1af1-743e-4dfe-b416-8774e21becc4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:24.580993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f1af1-743e-4dfe-b416-8774e21becc4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:24.581267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f1af1-743e-4dfe-b416-8774e21becc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:24.595554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47305154-a23a-4049-8805-b673ada29b01 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:24.599522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f1af1-743e-4dfe-b416-8774e21becc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-a93f1af1-743e-4dfe-b416-8774e21becc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:24.604717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:24.605666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:24.606414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 14/28] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:04:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:24.612047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f1af1-743e-4dfe-b416-8774e21becc4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:24.679226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:24.741975 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:24.757436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47305154-a23a-4049-8805-b673ada29b01 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:24.816492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f1af1-743e-4dfe-b416-8774e21becc4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:24.821512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1488e99-436f-43fb-aa86-f54b5c5ec1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:24.822386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1488e99-436f-43fb-aa86-f54b5c5ec1bf None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:24.822902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1488e99-436f-43fb-aa86-f54b5c5ec1bf None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:24.823392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1488e99-436f-43fb-aa86-f54b5c5ec1bf None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:24.824283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1488e99-436f-43fb-aa86-f54b5c5ec1bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:24.825890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1488e99-436f-43fb-aa86-f54b5c5ec1bf None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:24.855717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ef6e-e893-46b6-a1e4-87b204847f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:24.856328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ef6e-e893-46b6-a1e4-87b204847f27 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:24.856610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ef6e-e893-46b6-a1e4-87b204847f27 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:24.857085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ef6e-e893-46b6-a1e4-87b204847f27 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:24.857692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80ef6e-e893-46b6-a1e4-87b204847f27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:24.857998 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80ef6e-e893-46b6-a1e4-87b204847f27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:24.859703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80ef6e-e893-46b6-a1e4-87b204847f27 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:24.869940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 15/29] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:24.875301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ef6e-e893-46b6-a1e4-87b204847f27 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:24.991880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ef6e-e893-46b6-a1e4-87b204847f27 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:25.019463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:25.028542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa041b2-208d-4b3b-8e85-4d316cdcf15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:25.029183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa041b2-208d-4b3b-8e85-4d316cdcf15b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.029462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa041b2-208d-4b3b-8e85-4d316cdcf15b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.029737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa041b2-208d-4b3b-8e85-4d316cdcf15b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.042216 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa041b2-208d-4b3b-8e85-4d316cdcf15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:25.043981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa041b2-208d-4b3b-8e85-4d316cdcf15b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-7fa041b2-208d-4b3b-8e85-4d316cdcf15b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:04:25.065663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:25.066624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:04:25.067432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:04:25.067432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:04:25.067432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:25.070245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 16/30] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:25.071831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22abf-e6e7-4291-9d3d-28e9a90bb3bf None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.072356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22abf-e6e7-4291-9d3d-28e9a90bb3bf None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.072854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22abf-e6e7-4291-9d3d-28e9a90bb3bf None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.074526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 17/31] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:25.074843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 15/32] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 22 17:04:24 2022] POST /identity/v3/roles => generated 219 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:04:25.075674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c22abf-e6e7-4291-9d3d-28e9a90bb3bf None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:25.076030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1488e99-436f-43fb-aa86-f54b5c5ec1bf None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:25.198213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1488e99-436f-43fb-aa86-f54b5c5ec1bf None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:25.199503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c22abf-e6e7-4291-9d3d-28e9a90bb3bf None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:25.207428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:25.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7d28c-dd18-4c4d-bd7c-6642d3faa912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:25.240626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea5269-af17-493b-ac92-257cb55965fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:25.241755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea5269-af17-493b-ac92-257cb55965fa None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.242098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea5269-af17-493b-ac92-257cb55965fa None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.242668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7d28c-dd18-4c4d-bd7c-6642d3faa912 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.243135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7d28c-dd18-4c4d-bd7c-6642d3faa912 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.243620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7d28c-dd18-4c4d-bd7c-6642d3faa912 None admin] PATH_INFO: `/v3/users/b8814a79ac534e6e8b81d70404045912` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.243961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea5269-af17-493b-ac92-257cb55965fa None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.244868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea5269-af17-493b-ac92-257cb55965fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:25.245184 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea5269-af17-493b-ac92-257cb55965fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:25.245794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7d28c-dd18-4c4d-bd7c-6642d3faa912 None admin] RBAC: Authorizing `identity:get_user(user_id=b8814a79ac534e6e8b81d70404045912)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:25.246811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea5269-af17-493b-ac92-257cb55965fa None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:25.247738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7d28c-dd18-4c4d-bd7c-6642d3faa912 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:25.250093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 18/33] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:25 2022] GET /identity/v3/users/b8814a79ac534e6e8b81d70404045912 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:25.255675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3e676-a4bd-4441-8f5a-ec4e9d4d494a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.256366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3e676-a4bd-4441-8f5a-ec4e9d4d494a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.257516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3e676-a4bd-4441-8f5a-ec4e9d4d494a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.261561 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 16/34] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:25.267336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea5269-af17-493b-ac92-257cb55965fa None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:25.279261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d3c3e676-a4bd-4441-8f5a-ec4e9d4d494a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:25.280742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d3c3e676-a4bd-4441-8f5a-ec4e9d4d494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:25.329182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:25.357926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 19/35] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:25.359181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3e676-a4bd-4441-8f5a-ec4e9d4d494a None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:25.388449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea5269-af17-493b-ac92-257cb55965fa None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:25.429348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c3fe-bb42-4569-a7c0-b529ee83c27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:25.430325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c3fe-bb42-4569-a7c0-b529ee83c27e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.430523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c3fe-bb42-4569-a7c0-b529ee83c27e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.430982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c3fe-bb42-4569-a7c0-b529ee83c27e None admin] PATH_INFO: `/v3/domains/default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.432554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7c3fe-bb42-4569-a7c0-b529ee83c27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:25.434289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7c3fe-bb42-4569-a7c0-b529ee83c27e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:25.436032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 17/36] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:25.442110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c3fe-bb42-4569-a7c0-b529ee83c27e None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:25.519017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3e676-a4bd-4441-8f5a-ec4e9d4d494a None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:25.520704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:25.580437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c05cbd-3d50-416a-b15a-5da6006e8dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:25.581075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c05cbd-3d50-416a-b15a-5da6006e8dd7 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.581386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c05cbd-3d50-416a-b15a-5da6006e8dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.581628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c05cbd-3d50-416a-b15a-5da6006e8dd7 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.582276 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c05cbd-3d50-416a-b15a-5da6006e8dd7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:25.584137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c05cbd-3d50-416a-b15a-5da6006e8dd7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:25.607418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c3fe-bb42-4569-a7c0-b529ee83c27e None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:25.652185 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de6055-ea2f-4236-8eff-cf3683a496e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:25.652812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6055-ea2f-4236-8eff-cf3683a496e3 None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.653097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6055-ea2f-4236-8eff-cf3683a496e3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.653379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6055-ea2f-4236-8eff-cf3683a496e3 None admin] PATH_INFO: `/v3/domains/default/users/b8814a79ac534e6e8b81d70404045912/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.655817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de6055-ea2f-4236-8eff-cf3683a496e3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b8814a79ac534e6e8b81d70404045912, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:25.657305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de6055-ea2f-4236-8eff-cf3683a496e3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:25.742209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 18/37] 10.222.0.20 () {58 vars in 1361 bytes} [Mon Aug 22 17:04:25 2022] PUT /identity/v3/domains/default/users/b8814a79ac534e6e8b81d70404045912/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:04:25.743942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3179d-8fd9-4627-bbd4-4690ef86f823 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.743942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3179d-8fd9-4627-bbd4-4690ef86f823 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.743942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3179d-8fd9-4627-bbd4-4690ef86f823 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.745160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 19/38] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:25.746273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3179d-8fd9-4627-bbd4-4690ef86f823 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:25.874123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:25.884292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3179d-8fd9-4627-bbd4-4690ef86f823 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:25.930543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 20/39] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:04:25 2022] POST /identity/v3/projects => generated 301 bytes in 571 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:04:25.932169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f66885-0cf4-41e9-857f-bc913cf5cf0a None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.932169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f66885-0cf4-41e9-857f-bc913cf5cf0a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.932330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f66885-0cf4-41e9-857f-bc913cf5cf0a None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.933818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 21/40] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:25.936866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba1985-4aac-49d7-9128-b7193b44a079 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.936866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba1985-4aac-49d7-9128-b7193b44a079 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.936866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba1985-4aac-49d7-9128-b7193b44a079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.937269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361833f7-e0ab-491f-a178-42e7f80d38a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:25.937269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361833f7-e0ab-491f-a178-42e7f80d38a9 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:25.937269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361833f7-e0ab-491f-a178-42e7f80d38a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:25.937533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361833f7-e0ab-491f-a178-42e7f80d38a9 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:25.940050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361833f7-e0ab-491f-a178-42e7f80d38a9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:25.940050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361833f7-e0ab-491f-a178-42e7f80d38a9 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:25.956544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-63ba1985-4aac-49d7-9128-b7193b44a079 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:25.958151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-63ba1985-4aac-49d7-9128-b7193b44a079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:25.987694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:26.035225 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 20/41] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 22 17:04:25 2022] POST /identity/v3/roles => generated 225 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:04:26.036502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb73c3-3738-4cf1-a236-ab2bdcf4c23b None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:26.036787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb73c3-3738-4cf1-a236-ab2bdcf4c23b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:26.037070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb73c3-3738-4cf1-a236-ab2bdcf4c23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:26.049125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 22/42] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:26.083837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8eeb73c3-3738-4cf1-a236-ab2bdcf4c23b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:26.085729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8eeb73c3-3738-4cf1-a236-ab2bdcf4c23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:26.180122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 21/43] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:26.367756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba1985-4aac-49d7-9128-b7193b44a079 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:26.458251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb73c3-3738-4cf1-a236-ab2bdcf4c23b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:26.480977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba1985-4aac-49d7-9128-b7193b44a079 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:26.524386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43e4bb-f8e3-4790-83cd-8ac3f78ce0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:26.525022 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43e4bb-f8e3-4790-83cd-8ac3f78ce0fe None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:26.525313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43e4bb-f8e3-4790-83cd-8ac3f78ce0fe None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:26.525586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43e4bb-f8e3-4790-83cd-8ac3f78ce0fe None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:26.526249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43e4bb-f8e3-4790-83cd-8ac3f78ce0fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:26.527817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43e4bb-f8e3-4790-83cd-8ac3f78ce0fe None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:26.548144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:26.566823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb73c3-3738-4cf1-a236-ab2bdcf4c23b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:26.606735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b2edd-a5c6-44c0-b22c-267d091cf402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:26.607399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2edd-a5c6-44c0-b22c-267d091cf402 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:26.607679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2edd-a5c6-44c0-b22c-267d091cf402 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:26.607950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2edd-a5c6-44c0-b22c-267d091cf402 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:26.609121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b2edd-a5c6-44c0-b22c-267d091cf402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:26.610624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b2edd-a5c6-44c0-b22c-267d091cf402 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:26.612283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 22/44] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:26.619670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b2edd-a5c6-44c0-b22c-267d091cf402 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:26.638561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 23/45] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 22 17:04:26 2022] POST /identity/v3/roles => generated 223 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:04:26.754166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b2edd-a5c6-44c0-b22c-267d091cf402 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:26.796436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56708b-baef-4079-ba54-0b08d7f4165d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:26.797050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56708b-baef-4079-ba54-0b08d7f4165d None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:26.797325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56708b-baef-4079-ba54-0b08d7f4165d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:26.797593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56708b-baef-4079-ba54-0b08d7f4165d None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:26.798252 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56708b-baef-4079-ba54-0b08d7f4165d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:26.800042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56708b-baef-4079-ba54-0b08d7f4165d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:26.911521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 23/46] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:04:26 2022] POST /identity/v3/projects => generated 312 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:04:27.739389 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b6940-a395-4fc5-961b-faddf84b9888 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:27.739941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b6940-a395-4fc5-961b-faddf84b9888 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:27.739941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b6940-a395-4fc5-961b-faddf84b9888 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:27.742077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 24/47] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:27.747201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a75e9-975c-4cbf-b05a-aa9652f4911b None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:27.747495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a75e9-975c-4cbf-b05a-aa9652f4911b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:27.747783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a75e9-975c-4cbf-b05a-aa9652f4911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:27.755020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:27.766604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0f6a75e9-975c-4cbf-b05a-aa9652f4911b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:27.771107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0f6a75e9-975c-4cbf-b05a-aa9652f4911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:27.842715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 24/48] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:28.113683 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b6940-a395-4fc5-961b-faddf84b9888 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:28.135582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:28.221649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b6940-a395-4fc5-961b-faddf84b9888 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:28.258593 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b2ef9-ba4b-4404-a7d1-d34307ba4a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:28.259238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b2ef9-ba4b-4404-a7d1-d34307ba4a47 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:28.259522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b2ef9-ba4b-4404-a7d1-d34307ba4a47 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:28.259801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b2ef9-ba4b-4404-a7d1-d34307ba4a47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:28.270603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b2ef9-ba4b-4404-a7d1-d34307ba4a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-515b2ef9-ba4b-4404-a7d1-d34307ba4a47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:28.275431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:28.276412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:28.277240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 25/49] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:04:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:28.282726 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a75e9-975c-4cbf-b05a-aa9652f4911b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:28.386127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a75e9-975c-4cbf-b05a-aa9652f4911b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:28.420159 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777d410-afec-4a64-b794-0eba89166c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:28.420778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777d410-afec-4a64-b794-0eba89166c42 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:28.421065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777d410-afec-4a64-b794-0eba89166c42 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:28.421335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777d410-afec-4a64-b794-0eba89166c42 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:28.422030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777d410-afec-4a64-b794-0eba89166c42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:28.422332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777d410-afec-4a64-b794-0eba89166c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:28.423751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777d410-afec-4a64-b794-0eba89166c42 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:28.425698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:28.437802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 25/50] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:28.443542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b2ef9-ba4b-4404-a7d1-d34307ba4a47 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:28.530120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:28.552390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b2ef9-ba4b-4404-a7d1-d34307ba4a47 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:28.589272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850ff6-39d6-48c1-a70a-702a70778bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:28.589903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850ff6-39d6-48c1-a70a-702a70778bda None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:28.590188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850ff6-39d6-48c1-a70a-702a70778bda None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:28.590465 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850ff6-39d6-48c1-a70a-702a70778bda None admin] PATH_INFO: `/v3/users/b8814a79ac534e6e8b81d70404045912` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:28.591635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e850ff6-39d6-48c1-a70a-702a70778bda None admin] RBAC: Authorizing `identity:get_user(user_id=b8814a79ac534e6e8b81d70404045912)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:28.593103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e850ff6-39d6-48c1-a70a-702a70778bda None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:28.594919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 26/51] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:28 2022] GET /identity/v3/users/b8814a79ac534e6e8b81d70404045912 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:28.601022 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777d410-afec-4a64-b794-0eba89166c42 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:28.708315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777d410-afec-4a64-b794-0eba89166c42 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:28.742860 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbc557-12db-4516-be5d-af355e093876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:28.743497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc557-12db-4516-be5d-af355e093876 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:28.743770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc557-12db-4516-be5d-af355e093876 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:28.744040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc557-12db-4516-be5d-af355e093876 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:28.745165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbc557-12db-4516-be5d-af355e093876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:28.746639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbc557-12db-4516-be5d-af355e093876 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:28.748210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 26/52] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:28.754039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850ff6-39d6-48c1-a70a-702a70778bda None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:28.842002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb793ae8-3ffb-4f2c-8a12-658f2a63ff0b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:28.842545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb793ae8-3ffb-4f2c-8a12-658f2a63ff0b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:28.842545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb793ae8-3ffb-4f2c-8a12-658f2a63ff0b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:28.843889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 27/53] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:04:28.849045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089af75-4fe3-4c11-bce8-5c7874b72c6c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:28.849330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089af75-4fe3-4c11-bce8-5c7874b72c6c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:28.849606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089af75-4fe3-4c11-bce8-5c7874b72c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:28.857682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850ff6-39d6-48c1-a70a-702a70778bda None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:28.867931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0089af75-4fe3-4c11-bce8-5c7874b72c6c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:28.869140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0089af75-4fe3-4c11-bce8-5c7874b72c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:28.892342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762097d1-5f65-4fe4-ac8d-34c43454bbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:28.892945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762097d1-5f65-4fe4-ac8d-34c43454bbed None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:28.893227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762097d1-5f65-4fe4-ac8d-34c43454bbed None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:28.893501 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762097d1-5f65-4fe4-ac8d-34c43454bbed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:28.894233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762097d1-5f65-4fe4-ac8d-34c43454bbed None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.domain.id=default, user.id=b8814a79ac534e6e8b81d70404045912) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:28.894567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762097d1-5f65-4fe4-ac8d-34c43454bbed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:28.896039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762097d1-5f65-4fe4-ac8d-34c43454bbed None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:28.902885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 27/54] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Aug 22 17:04:28 2022] GET /identity/v3/role_assignments?user.id=b8814a79ac534e6e8b81d70404045912&scope.domain.id=default&role.id=4b10782d99b74f5bb453b82286012e0c => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:28.931947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:28.940925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 28/55] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:04:29.215465 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762097d1-5f65-4fe4-ac8d-34c43454bbed None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:29.228508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:29.317827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762097d1-5f65-4fe4-ac8d-34c43454bbed None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:29.353048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141108e7-46be-417b-bac1-8e1b97f2b8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:29.353675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141108e7-46be-417b-bac1-8e1b97f2b8fc None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:29.353952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141108e7-46be-417b-bac1-8e1b97f2b8fc None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:29.354220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141108e7-46be-417b-bac1-8e1b97f2b8fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:29.355356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141108e7-46be-417b-bac1-8e1b97f2b8fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:29.356825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141108e7-46be-417b-bac1-8e1b97f2b8fc None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:29.358389 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 28/56] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:29.364060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089af75-4fe3-4c11-bce8-5c7874b72c6c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:29.464098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089af75-4fe3-4c11-bce8-5c7874b72c6c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:29.603619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83397a-570e-4954-8961-01b4166e5ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:29.604148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83397a-570e-4954-8961-01b4166e5ee1 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:29.604437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83397a-570e-4954-8961-01b4166e5ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:29.604709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83397a-570e-4954-8961-01b4166e5ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:29.605400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83397a-570e-4954-8961-01b4166e5ee1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:29.606921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83397a-570e-4954-8961-01b4166e5ee1 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:29.743284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 29/57] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Aug 22 17:04:29 2022] POST /identity/v3/projects => generated 298 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:04:31.685044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a69d1-6830-42d3-874e-ec5187108564 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:31.685555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a69d1-6830-42d3-874e-ec5187108564 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:31.685639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a69d1-6830-42d3-874e-ec5187108564 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:31.686984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 29/58] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:31.692207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494918f-92be-48f0-9726-7be985c34afc None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:31.692644 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494918f-92be-48f0-9726-7be985c34afc None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:31.692767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494918f-92be-48f0-9726-7be985c34afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:31.711758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f494918f-92be-48f0-9726-7be985c34afc None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:31.713018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f494918f-92be-48f0-9726-7be985c34afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:31.755091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:31.780298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 30/59] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:32.050630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a69d1-6830-42d3-874e-ec5187108564 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:32.131348 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:32.154714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a69d1-6830-42d3-874e-ec5187108564 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:32.191183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62de799-107d-484f-a7d0-f8ec610f8ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:32.191811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62de799-107d-484f-a7d0-f8ec610f8ad6 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:32.192088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62de799-107d-484f-a7d0-f8ec610f8ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:32.192358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62de799-107d-484f-a7d0-f8ec610f8ad6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:32.193451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62de799-107d-484f-a7d0-f8ec610f8ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:32.196192 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62de799-107d-484f-a7d0-f8ec610f8ad6 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:32.197773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 30/60] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:32.203437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494918f-92be-48f0-9726-7be985c34afc None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:32.305569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494918f-92be-48f0-9726-7be985c34afc None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:32.345320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3f574-d6fa-4a1b-a79d-ccedbf7f7037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:32.345946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3f574-d6fa-4a1b-a79d-ccedbf7f7037 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:32.346229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3f574-d6fa-4a1b-a79d-ccedbf7f7037 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:32.346533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3f574-d6fa-4a1b-a79d-ccedbf7f7037 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:32.347201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3f574-d6fa-4a1b-a79d-ccedbf7f7037 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:32.348662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3f574-d6fa-4a1b-a79d-ccedbf7f7037 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:32.484000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 31/61] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:04:32 2022] POST /identity/v3/users => generated 275 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:04:35.619671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c279f-d144-42e8-9686-67a99a112b7e None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:35.619671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c279f-d144-42e8-9686-67a99a112b7e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:35.619671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c279f-d144-42e8-9686-67a99a112b7e None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:35.619671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 31/62] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:35.625562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6129aea-09b9-4888-b45d-5aea1398d4e3 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:35.625978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6129aea-09b9-4888-b45d-5aea1398d4e3 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:35.626106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6129aea-09b9-4888-b45d-5aea1398d4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:35.655941 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c6129aea-09b9-4888-b45d-5aea1398d4e3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:35.657381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c6129aea-09b9-4888-b45d-5aea1398d4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:35.696795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:35.755356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 32/63] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:35.872616 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936e753-c85d-4ed0-9a8d-8e82a402488b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:35.873126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936e753-c85d-4ed0-9a8d-8e82a402488b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:35.873560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936e753-c85d-4ed0-9a8d-8e82a402488b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:35.875371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 32/64] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:35.880486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfdc62-494d-4f00-8c82-e68786c21d0c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:35.880777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfdc62-494d-4f00-8c82-e68786c21d0c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:35.881774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfdc62-494d-4f00-8c82-e68786c21d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:35.912008 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d1dfdc62-494d-4f00-8c82-e68786c21d0c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:35.913800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d1dfdc62-494d-4f00-8c82-e68786c21d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:35.980395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885893c6-08c8-42fc-80f0-1e6973ae9d0d None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:35.981020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885893c6-08c8-42fc-80f0-1e6973ae9d0d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:35.981483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885893c6-08c8-42fc-80f0-1e6973ae9d0d None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:35.983065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 33/65] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:04:35.988515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74ee42-b7bc-4b70-b6c3-c8acaa94044c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:35.989041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74ee42-b7bc-4b70-b6c3-c8acaa94044c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:35.989524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74ee42-b7bc-4b70-b6c3-c8acaa94044c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.019164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-be74ee42-b7bc-4b70-b6c3-c8acaa94044c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:36.019728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 33/66] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:36.021193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-be74ee42-b7bc-4b70-b6c3-c8acaa94044c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:36.076149 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a97328-185f-460c-8d44-f5b8d03d161a None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.076773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a97328-185f-460c-8d44-f5b8d03d161a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.077241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a97328-185f-460c-8d44-f5b8d03d161a None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.078824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 34/67] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:04:36.084201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf586c66-94c2-4062-a302-8fa549b2509f None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.085006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf586c66-94c2-4062-a302-8fa549b2509f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.085480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf586c66-94c2-4062-a302-8fa549b2509f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.109321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-bf586c66-94c2-4062-a302-8fa549b2509f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:36.111403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bf586c66-94c2-4062-a302-8fa549b2509f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:36.125886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 34/68] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:36.126809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74ee42-b7bc-4b70-b6c3-c8acaa94044c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:36.208869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 35/69] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 22 17:04:36.210759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94b04f-54b8-4583-a651-a9eb454753d3 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.211048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94b04f-54b8-4583-a651-a9eb454753d3 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.211378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94b04f-54b8-4583-a651-a9eb454753d3 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.212889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 36/70] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:36.218445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b1ea0-a689-472b-adf6-2b0dbf4a5092 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.218752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b1ea0-a689-472b-adf6-2b0dbf4a5092 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.219032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b1ea0-a689-472b-adf6-2b0dbf4a5092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.250726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:36.252354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-151b1ea0-a689-472b-adf6-2b0dbf4a5092 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:36.253721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-151b1ea0-a689-472b-adf6-2b0dbf4a5092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:36.274078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74ee42-b7bc-4b70-b6c3-c8acaa94044c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:36.292515 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:36.311835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e893-ef00-4581-aa35-e68881844a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:36.312567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e893-ef00-4581-aa35-e68881844a8a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.312936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e893-ef00-4581-aa35-e68881844a8a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.313278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e893-ef00-4581-aa35-e68881844a8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.325779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3e893-ef00-4581-aa35-e68881844a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-21d3e893-ef00-4581-aa35-e68881844a8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:36.331035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:36.331926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:36.332717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 35/71] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:36.339076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e893-ef00-4581-aa35-e68881844a8a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:36.382121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 37/72] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:36.383239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b1ea0-a689-472b-adf6-2b0dbf4a5092 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:36.502424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e893-ef00-4581-aa35-e68881844a8a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:36.539075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bf6d0-662c-4f56-8ab6-0c3df808b3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:36.539730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf6d0-662c-4f56-8ab6-0c3df808b3f0 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.540012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf6d0-662c-4f56-8ab6-0c3df808b3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.540282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf6d0-662c-4f56-8ab6-0c3df808b3f0 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.540962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bf6d0-662c-4f56-8ab6-0c3df808b3f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:36.541256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bf6d0-662c-4f56-8ab6-0c3df808b3f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:36.542668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bf6d0-662c-4f56-8ab6-0c3df808b3f0 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:36.543863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b1ea0-a689-472b-adf6-2b0dbf4a5092 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:36.545635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:36.554189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 36/73] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:36.554889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bf6d0-662c-4f56-8ab6-0c3df808b3f0 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:36.595274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd10c1-1449-4bc0-9d3a-723370397d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:36.597138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd10c1-1449-4bc0-9d3a-723370397d29 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.597625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd10c1-1449-4bc0-9d3a-723370397d29 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.598096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd10c1-1449-4bc0-9d3a-723370397d29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.614664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd10c1-1449-4bc0-9d3a-723370397d29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-7bbd10c1-1449-4bc0-9d3a-723370397d29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:36.622850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:36.623699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 38/74] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:04:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:36.629869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd10c1-1449-4bc0-9d3a-723370397d29 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:36.673806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:36.683289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bf6d0-662c-4f56-8ab6-0c3df808b3f0 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:36.718286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e528f1-102a-4a8c-a82b-d7289e6f55a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:36.718971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e528f1-102a-4a8c-a82b-d7289e6f55a9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.719260 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e528f1-102a-4a8c-a82b-d7289e6f55a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.719533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e528f1-102a-4a8c-a82b-d7289e6f55a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.730601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e528f1-102a-4a8c-a82b-d7289e6f55a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:36.731737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd10c1-1449-4bc0-9d3a-723370397d29 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-e6e528f1-102a-4a8c-a82b-d7289e6f55a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:36.735868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:36.736745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:36.737741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 37/75] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:36.742934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e528f1-102a-4a8c-a82b-d7289e6f55a9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:36.767510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431f10a-5ae8-4329-9110-324459a9e3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:36.768150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431f10a-5ae8-4329-9110-324459a9e3ad None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.768428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431f10a-5ae8-4329-9110-324459a9e3ad None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.768708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431f10a-5ae8-4329-9110-324459a9e3ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.779304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431f10a-5ae8-4329-9110-324459a9e3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-2431f10a-5ae8-4329-9110-324459a9e3ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:36.784173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:36.785151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 39/76] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:04:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:36.787794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431f10a-5ae8-4329-9110-324459a9e3ad None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:36.807066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:36.848496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e528f1-102a-4a8c-a82b-d7289e6f55a9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:36.859663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:36.886094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47bb63-c7da-4e90-839c-426f3de77cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:36.912826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47bb63-c7da-4e90-839c-426f3de77cc5 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.912826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47bb63-c7da-4e90-839c-426f3de77cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.912826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47bb63-c7da-4e90-839c-426f3de77cc5 None admin] PATH_INFO: `/v3/users/734f29275e5547fcb788acba34124924` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.913131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431f10a-5ae8-4329-9110-324459a9e3ad None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:36.913226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47bb63-c7da-4e90-839c-426f3de77cc5 None admin] RBAC: Authorizing `identity:get_user(user_id=734f29275e5547fcb788acba34124924)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:36.913226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47bb63-c7da-4e90-839c-426f3de77cc5 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:36.913226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 38/77] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:36 2022] GET /identity/v3/users/734f29275e5547fcb788acba34124924 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:36.913226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47bb63-c7da-4e90-839c-426f3de77cc5 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:36.922982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9ae33-d2bb-4bc4-ab13-f70c3a431d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:36.923592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9ae33-d2bb-4bc4-ab13-f70c3a431d33 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:36.923917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9ae33-d2bb-4bc4-ab13-f70c3a431d33 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:36.924390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9ae33-d2bb-4bc4-ab13-f70c3a431d33 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:36.925045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9ae33-d2bb-4bc4-ab13-f70c3a431d33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:36.925359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9ae33-d2bb-4bc4-ab13-f70c3a431d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:36.926745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9ae33-d2bb-4bc4-ab13-f70c3a431d33 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:36.935507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 40/78] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:36.940028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9ae33-d2bb-4bc4-ab13-f70c3a431d33 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:37.011206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47bb63-c7da-4e90-839c-426f3de77cc5 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:37.028177 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:37.039732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9ae33-d2bb-4bc4-ab13-f70c3a431d33 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:37.045165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe87ec-9af2-4e1e-8619-1efb87c13b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:37.045790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe87ec-9af2-4e1e-8619-1efb87c13b39 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:37.046068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe87ec-9af2-4e1e-8619-1efb87c13b39 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:37.046340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe87ec-9af2-4e1e-8619-1efb87c13b39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:37.047466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe87ec-9af2-4e1e-8619-1efb87c13b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:37.048914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe87ec-9af2-4e1e-8619-1efb87c13b39 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:37.050766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 39/79] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:37.051529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe87ec-9af2-4e1e-8619-1efb87c13b39 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:37.077563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9467c-303f-4a3f-a18c-eaeb96474586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:37.078172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9467c-303f-4a3f-a18c-eaeb96474586 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:37.078455 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9467c-303f-4a3f-a18c-eaeb96474586 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:37.078776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9467c-303f-4a3f-a18c-eaeb96474586 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:37.079439 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9467c-303f-4a3f-a18c-eaeb96474586 None admin] RBAC: Adding query filter params (name=member) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:37.079741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9467c-303f-4a3f-a18c-eaeb96474586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:37.081113 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9467c-303f-4a3f-a18c-eaeb96474586 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:37.082446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:37.090800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 41/80] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:37.095604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9467c-303f-4a3f-a18c-eaeb96474586 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:37.156806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe87ec-9af2-4e1e-8619-1efb87c13b39 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:37.189299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9467c-303f-4a3f-a18c-eaeb96474586 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:37.192228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df4d58-c510-4b3f-a95a-de48c98d317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:37.192837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df4d58-c510-4b3f-a95a-de48c98d317d None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:37.193118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df4d58-c510-4b3f-a95a-de48c98d317d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:37.193444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df4d58-c510-4b3f-a95a-de48c98d317d None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:37.194116 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df4d58-c510-4b3f-a95a-de48c98d317d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:37.194422 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df4d58-c510-4b3f-a95a-de48c98d317d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:37.195839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df4d58-c510-4b3f-a95a-de48c98d317d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:37.203852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 40/81] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:04:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:37.204669 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df4d58-c510-4b3f-a95a-de48c98d317d None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:37.230371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94767fae-5523-4263-a1d7-7f9dcde2a34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:37.231100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94767fae-5523-4263-a1d7-7f9dcde2a34d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:37.231287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94767fae-5523-4263-a1d7-7f9dcde2a34d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:37.231579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94767fae-5523-4263-a1d7-7f9dcde2a34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:37.254366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:37.295456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:37.302698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94767fae-5523-4263-a1d7-7f9dcde2a34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:37.304309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94767fae-5523-4263-a1d7-7f9dcde2a34d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:37.310287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df4d58-c510-4b3f-a95a-de48c98d317d None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:37.346041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597d2c0-c804-4773-b6a5-aca6f5145d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:37.346710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597d2c0-c804-4773-b6a5-aca6f5145d70 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:37.346989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597d2c0-c804-4773-b6a5-aca6f5145d70 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:37.347262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597d2c0-c804-4773-b6a5-aca6f5145d70 None admin] PATH_INFO: `/v3/users/b8814a79ac534e6e8b81d70404045912` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:37.348494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597d2c0-c804-4773-b6a5-aca6f5145d70 None admin] RBAC: Authorizing `identity:get_user(user_id=b8814a79ac534e6e8b81d70404045912)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:37.350803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597d2c0-c804-4773-b6a5-aca6f5145d70 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:37.352384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 41/82] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:37 2022] GET /identity/v3/users/b8814a79ac534e6e8b81d70404045912 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:37.353202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597d2c0-c804-4773-b6a5-aca6f5145d70 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:37.406699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 42/83] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:37.407393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94767fae-5523-4263-a1d7-7f9dcde2a34d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:37.454375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597d2c0-c804-4773-b6a5-aca6f5145d70 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:37.468844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:37.488338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa629a-5b29-4c0e-bd29-c530847e7768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:37.488942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa629a-5b29-4c0e-bd29-c530847e7768 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:37.489232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa629a-5b29-4c0e-bd29-c530847e7768 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:37.491902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa629a-5b29-4c0e-bd29-c530847e7768 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:37.491902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa629a-5b29-4c0e-bd29-c530847e7768 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:37.491902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa629a-5b29-4c0e-bd29-c530847e7768 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:37.505548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:37.512210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94767fae-5523-4263-a1d7-7f9dcde2a34d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:37.546072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86faad4-c440-49bc-8df8-03c832a0f369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:37.546746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86faad4-c440-49bc-8df8-03c832a0f369 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:37.547031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86faad4-c440-49bc-8df8-03c832a0f369 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:37.547308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86faad4-c440-49bc-8df8-03c832a0f369 None admin] PATH_INFO: `/v3/users/734f29275e5547fcb788acba34124924` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:37.548457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86faad4-c440-49bc-8df8-03c832a0f369 None admin] RBAC: Authorizing `identity:get_user(user_id=734f29275e5547fcb788acba34124924)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:37.550175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86faad4-c440-49bc-8df8-03c832a0f369 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:37.551783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 43/84] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:37 2022] GET /identity/v3/users/734f29275e5547fcb788acba34124924 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:37.552569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86faad4-c440-49bc-8df8-03c832a0f369 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:37.638299 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 42/85] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:04:37 2022] POST /identity/v3/users => generated 289 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:04:37.643618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa629a-5b29-4c0e-bd29-c530847e7768 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:37.653122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86faad4-c440-49bc-8df8-03c832a0f369 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:37.695677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6e435-e6f4-4f86-b6cf-7a7435ea3be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:37.696376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e435-e6f4-4f86-b6cf-7a7435ea3be2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:37.696663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e435-e6f4-4f86-b6cf-7a7435ea3be2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:37.696937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e435-e6f4-4f86-b6cf-7a7435ea3be2 None admin] PATH_INFO: `/v3/users/734f29275e5547fcb788acba34124924` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:37.698165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6e435-e6f4-4f86-b6cf-7a7435ea3be2 None admin] RBAC: Authorizing `identity:get_user(user_id=734f29275e5547fcb788acba34124924)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:37.699903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6e435-e6f4-4f86-b6cf-7a7435ea3be2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:37.701984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 44/86] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:37 2022] GET /identity/v3/users/734f29275e5547fcb788acba34124924 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:37.705393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6e435-e6f4-4f86-b6cf-7a7435ea3be2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:37.740482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:37.766376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa629a-5b29-4c0e-bd29-c530847e7768 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:37.812306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74970662-7387-4759-9639-2f0ab9465bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:37.812928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74970662-7387-4759-9639-2f0ab9465bff None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:37.813212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74970662-7387-4759-9639-2f0ab9465bff None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:37.813486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74970662-7387-4759-9639-2f0ab9465bff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:37.824169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6e435-e6f4-4f86-b6cf-7a7435ea3be2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:37.863774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:37.868387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456168a-f04d-42d0-9540-47894a1cfd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:37.869350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456168a-f04d-42d0-9540-47894a1cfd7d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:37.869776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456168a-f04d-42d0-9540-47894a1cfd7d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:37.880589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456168a-f04d-42d0-9540-47894a1cfd7d None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:37.880589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456168a-f04d-42d0-9540-47894a1cfd7d None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:37.880589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456168a-f04d-42d0-9540-47894a1cfd7d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:37.880589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 45/87] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:37 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:37.880589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456168a-f04d-42d0-9540-47894a1cfd7d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:37.909091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74970662-7387-4759-9639-2f0ab9465bff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:37.910826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74970662-7387-4759-9639-2f0ab9465bff None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:38.007673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:38.010806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456168a-f04d-42d0-9540-47894a1cfd7d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:38.039471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 43/88] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:38.040361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74970662-7387-4759-9639-2f0ab9465bff None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:38.058138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738cad9-c833-4528-ba17-acc19dbdffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:38.059064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738cad9-c833-4528-ba17-acc19dbdffa9 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:38.059352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738cad9-c833-4528-ba17-acc19dbdffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:38.059895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738cad9-c833-4528-ba17-acc19dbdffa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:38.122796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:38.162883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74970662-7387-4759-9639-2f0ab9465bff None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:38.188890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738cad9-c833-4528-ba17-acc19dbdffa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:38.190169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738cad9-c833-4528-ba17-acc19dbdffa9 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:38.207157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad13ab-d913-4f58-8280-0a00a561b3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:38.208092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad13ab-d913-4f58-8280-0a00a561b3c8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:38.208528 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad13ab-d913-4f58-8280-0a00a561b3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:38.209543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad13ab-d913-4f58-8280-0a00a561b3c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:38.303806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad13ab-d913-4f58-8280-0a00a561b3c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:38.305569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad13ab-d913-4f58-8280-0a00a561b3c8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:38.362063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:38.370006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 46/89] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:38.371363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738cad9-c833-4528-ba17-acc19dbdffa9 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:38.391404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:38.432564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 44/90] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:38.433883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad13ab-d913-4f58-8280-0a00a561b3c8 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:38.555779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad13ab-d913-4f58-8280-0a00a561b3c8 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:38.573481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738cad9-c833-4528-ba17-acc19dbdffa9 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:38.618627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b7247-e5d9-4cd4-8cf0-4151a31dce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:38.619631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b7247-e5d9-4cd4-8cf0-4151a31dce4c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:38.620775 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b7247-e5d9-4cd4-8cf0-4151a31dce4c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:38.622598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b7247-e5d9-4cd4-8cf0-4151a31dce4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:38.623609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b7247-e5d9-4cd4-8cf0-4151a31dce4c None admin] RBAC: Adding query filter params (role.id=13accf89c12040508974db47c6905b31, scope.project.id=c84516b7267f4b43bf816381e3e60ae6, user.id=734f29275e5547fcb788acba34124924) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:38.625405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b7247-e5d9-4cd4-8cf0-4151a31dce4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:38.627152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b7247-e5d9-4cd4-8cf0-4151a31dce4c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:38.642505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a74eb-3f9c-46d5-a60a-1bdb07d1abe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:38.642505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a74eb-3f9c-46d5-a60a-1bdb07d1abe3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:38.642505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a74eb-3f9c-46d5-a60a-1bdb07d1abe3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:38.642505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a74eb-3f9c-46d5-a60a-1bdb07d1abe3 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:38.642899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a74eb-3f9c-46d5-a60a-1bdb07d1abe3 None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:38.644867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 47/91] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:38 2022] GET /identity/v3/role_assignments?user.id=734f29275e5547fcb788acba34124924&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=13accf89c12040508974db47c6905b31 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:38.647009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a74eb-3f9c-46d5-a60a-1bdb07d1abe3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:38.648210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b7247-e5d9-4cd4-8cf0-4151a31dce4c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:38.650365 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 45/92] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:38 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:38.651449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a74eb-3f9c-46d5-a60a-1bdb07d1abe3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:38.758517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:38.820404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:38.894258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b7247-e5d9-4cd4-8cf0-4151a31dce4c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:38.901680 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a74eb-3f9c-46d5-a60a-1bdb07d1abe3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:38.941969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be629c2-9b8d-45b5-812c-caa25a03024d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:38.942731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be629c2-9b8d-45b5-812c-caa25a03024d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:38.943675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be629c2-9b8d-45b5-812c-caa25a03024d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:38.944063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be629c2-9b8d-45b5-812c-caa25a03024d None admin] PATH_INFO: `/v3/projects/3cd1942bafc649eaa8bbd93187a3a618` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:38.945648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be629c2-9b8d-45b5-812c-caa25a03024d None admin] RBAC: Authorizing `identity:get_project(project_id=3cd1942bafc649eaa8bbd93187a3a618)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:38.952491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be629c2-9b8d-45b5-812c-caa25a03024d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:38.954431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 48/93] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:38 2022] GET /identity/v3/projects/3cd1942bafc649eaa8bbd93187a3a618 => generated 312 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:38.955367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be629c2-9b8d-45b5-812c-caa25a03024d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:38.956524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af470515-3d59-4baf-8700-fd32e0f16cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:38.957190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af470515-3d59-4baf-8700-fd32e0f16cf2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:38.957473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af470515-3d59-4baf-8700-fd32e0f16cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:38.957749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af470515-3d59-4baf-8700-fd32e0f16cf2 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:38.960370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af470515-3d59-4baf-8700-fd32e0f16cf2 None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:38.962008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af470515-3d59-4baf-8700-fd32e0f16cf2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:38.963818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 46/94] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:38 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:38.964649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af470515-3d59-4baf-8700-fd32e0f16cf2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:39.126811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af470515-3d59-4baf-8700-fd32e0f16cf2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:39.135390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:39.169227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec38c3-26b8-4099-a924-8754eeff3aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:39.169884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec38c3-26b8-4099-a924-8754eeff3aaa None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:39.170002 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec38c3-26b8-4099-a924-8754eeff3aaa None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:39.173228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec38c3-26b8-4099-a924-8754eeff3aaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:39.173228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec38c3-26b8-4099-a924-8754eeff3aaa None admin] RBAC: Adding query filter params (role.id=13accf89c12040508974db47c6905b31, scope.project.id=3cd1942bafc649eaa8bbd93187a3a618, user.id=734f29275e5547fcb788acba34124924) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:39.173228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec38c3-26b8-4099-a924-8754eeff3aaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:39.173228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec38c3-26b8-4099-a924-8754eeff3aaa None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:39.181016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 47/95] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:38 2022] GET /identity/v3/role_assignments?user.id=734f29275e5547fcb788acba34124924&scope.project.id=3cd1942bafc649eaa8bbd93187a3a618&role.id=13accf89c12040508974db47c6905b31 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:39.185678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec38c3-26b8-4099-a924-8754eeff3aaa None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:39.211384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be629c2-9b8d-45b5-812c-caa25a03024d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:39.272836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:39.280399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03afc9-fc98-4b46-9ece-6dcb2224bf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:39.281025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03afc9-fc98-4b46-9ece-6dcb2224bf7c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:39.281306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03afc9-fc98-4b46-9ece-6dcb2224bf7c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:39.281580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03afc9-fc98-4b46-9ece-6dcb2224bf7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:39.282325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03afc9-fc98-4b46-9ece-6dcb2224bf7c None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=c84516b7267f4b43bf816381e3e60ae6, user.id=b8814a79ac534e6e8b81d70404045912) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:39.282715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03afc9-fc98-4b46-9ece-6dcb2224bf7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:39.291140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03afc9-fc98-4b46-9ece-6dcb2224bf7c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:39.295533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 49/96] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:38 2022] GET /identity/v3/role_assignments?user.id=b8814a79ac534e6e8b81d70404045912&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=4b10782d99b74f5bb453b82286012e0c => generated 259 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:39.360703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec38c3-26b8-4099-a924-8754eeff3aaa None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:39.412509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a18bf-f2e7-49a8-98e1-8ff0c7e67018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:39.413190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a18bf-f2e7-49a8-98e1-8ff0c7e67018 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:39.413477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a18bf-f2e7-49a8-98e1-8ff0c7e67018 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:39.413751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a18bf-f2e7-49a8-98e1-8ff0c7e67018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:39.414520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a18bf-f2e7-49a8-98e1-8ff0c7e67018 None admin] RBAC: Adding query filter params (role.id=c2fe89b1b9fa47b19e02efd64ace9000, scope.project.id=c84516b7267f4b43bf816381e3e60ae6, user.id=734f29275e5547fcb788acba34124924) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:39.414832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a18bf-f2e7-49a8-98e1-8ff0c7e67018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:39.417017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a18bf-f2e7-49a8-98e1-8ff0c7e67018 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:39.427353 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 48/97] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:39 2022] GET /identity/v3/role_assignments?user.id=734f29275e5547fcb788acba34124924&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=c2fe89b1b9fa47b19e02efd64ace9000 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:40.102549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b3fe9-06d8-46e6-b08f-37197b8d815b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:40.103557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b3fe9-06d8-46e6-b08f-37197b8d815b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:40.104085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b3fe9-06d8-46e6-b08f-37197b8d815b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:40.105687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 50/98] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:40.120213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ab7e0-c953-4b54-93e7-d1b51bf72565 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:40.121403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ab7e0-c953-4b54-93e7-d1b51bf72565 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:40.121893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ab7e0-c953-4b54-93e7-d1b51bf72565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:40.151498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b19ab7e0-c953-4b54-93e7-d1b51bf72565 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:40.153194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b19ab7e0-c953-4b54-93e7-d1b51bf72565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:40.175950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58832775-0f14-4d93-9ccb-1f6283201af1 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:40.176567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58832775-0f14-4d93-9ccb-1f6283201af1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:40.177045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58832775-0f14-4d93-9ccb-1f6283201af1 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:40.179347 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 51/99] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:04:40.185710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d05f4-48fa-4f93-8217-a28a65c2815a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:40.186262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d05f4-48fa-4f93-8217-a28a65c2815a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:40.186809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d05f4-48fa-4f93-8217-a28a65c2815a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:40.205046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:40.213199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-4a4d05f4-48fa-4f93-8217-a28a65c2815a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:40.214543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4a4d05f4-48fa-4f93-8217-a28a65c2815a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:40.284690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 49/100] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:40.332397 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 52/101] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:04:40.646064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ab7e0-c953-4b54-93e7-d1b51bf72565 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:40.754726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ab7e0-c953-4b54-93e7-d1b51bf72565 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:40.798497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1763-210c-4e11-b7eb-3a4825d0d899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:40.799459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1763-210c-4e11-b7eb-3a4825d0d899 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:40.799739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1763-210c-4e11-b7eb-3a4825d0d899 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:40.800013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1763-210c-4e11-b7eb-3a4825d0d899 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:40.817227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1763-210c-4e11-b7eb-3a4825d0d899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-6c8b1763-210c-4e11-b7eb-3a4825d0d899 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:40.823601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:40.824472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:40.825312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 50/102] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:40.831885 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d05f4-48fa-4f93-8217-a28a65c2815a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:40.855362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1763-210c-4e11-b7eb-3a4825d0d899 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:40.888028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:41.019388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d05f4-48fa-4f93-8217-a28a65c2815a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:41.036012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1763-210c-4e11-b7eb-3a4825d0d899 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:41.075258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:41.095944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86451e3d-ade5-4d7c-a4a4-4a0f5724036c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:41.096508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86451e3d-ade5-4d7c-a4a4-4a0f5724036c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:41.096790 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86451e3d-ade5-4d7c-a4a4-4a0f5724036c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:41.097064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86451e3d-ade5-4d7c-a4a4-4a0f5724036c None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:41.097751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86451e3d-ade5-4d7c-a4a4-4a0f5724036c None admin] RBAC: Adding query filter params (name=reader) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:41.098053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86451e3d-ade5-4d7c-a4a4-4a0f5724036c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:41.099533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86451e3d-ade5-4d7c-a4a4-4a0f5724036c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:41.112927 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 53/103] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:41.114180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444ff5c-f23f-447e-a518-4af90ae2d4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:41.115794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444ff5c-f23f-447e-a518-4af90ae2d4b4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:41.116234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444ff5c-f23f-447e-a518-4af90ae2d4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:41.116668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444ff5c-f23f-447e-a518-4af90ae2d4b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:41.117960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444ff5c-f23f-447e-a518-4af90ae2d4b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:41.119740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444ff5c-f23f-447e-a518-4af90ae2d4b4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:41.120787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86451e3d-ade5-4d7c-a4a4-4a0f5724036c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:41.122978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 51/104] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:41.133740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444ff5c-f23f-447e-a518-4af90ae2d4b4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:41.287480 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:41.307042 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444ff5c-f23f-447e-a518-4af90ae2d4b4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:41.323190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86451e3d-ade5-4d7c-a4a4-4a0f5724036c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:41.368753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c8cc6-c8b7-4907-9e50-a6ec7f893f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:41.369821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c8cc6-c8b7-4907-9e50-a6ec7f893f2e None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:41.370329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c8cc6-c8b7-4907-9e50-a6ec7f893f2e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:41.370828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c8cc6-c8b7-4907-9e50-a6ec7f893f2e None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:41.371684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c8cc6-c8b7-4907-9e50-a6ec7f893f2e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:41.373368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c8cc6-c8b7-4907-9e50-a6ec7f893f2e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:41.417125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282bf8c-dc17-48ed-8644-0ede8c1c7e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:41.421563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282bf8c-dc17-48ed-8644-0ede8c1c7e9c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:41.421563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282bf8c-dc17-48ed-8644-0ede8c1c7e9c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:41.421563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282bf8c-dc17-48ed-8644-0ede8c1c7e9c None admin] PATH_INFO: `/v3/users/f766fe03e07644c5b1f918e54a832d4d` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:41.443783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282bf8c-dc17-48ed-8644-0ede8c1c7e9c None admin] RBAC: Authorizing `identity:get_user(user_id=f766fe03e07644c5b1f918e54a832d4d)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:41.445617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282bf8c-dc17-48ed-8644-0ede8c1c7e9c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:41.447558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 54/105] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:41 2022] GET /identity/v3/users/f766fe03e07644c5b1f918e54a832d4d => generated 289 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:41.453584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282bf8c-dc17-48ed-8644-0ede8c1c7e9c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:41.475696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 52/106] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:04:41 2022] POST /identity/v3/projects => generated 302 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:04:41.478690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1076c-8853-4e47-94b8-d3c0289de9a7 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:41.478690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1076c-8853-4e47-94b8-d3c0289de9a7 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:41.478690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1076c-8853-4e47-94b8-d3c0289de9a7 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:41.481652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 53/107] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:41.486451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bd380-cb1d-4b88-9115-b1066b8ffccc None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:41.487935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bd380-cb1d-4b88-9115-b1066b8ffccc None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:41.488448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bd380-cb1d-4b88-9115-b1066b8ffccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:41.532615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-860bd380-cb1d-4b88-9115-b1066b8ffccc None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:41.537962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-860bd380-cb1d-4b88-9115-b1066b8ffccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:41.573816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:41.707866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 54/108] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:41.717547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282bf8c-dc17-48ed-8644-0ede8c1c7e9c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:41.764021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794165a-1b61-4937-9f23-74f2ef65734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:41.764651 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794165a-1b61-4937-9f23-74f2ef65734c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:41.764937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794165a-1b61-4937-9f23-74f2ef65734c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:41.765215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794165a-1b61-4937-9f23-74f2ef65734c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:41.854273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794165a-1b61-4937-9f23-74f2ef65734c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:41.855731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794165a-1b61-4937-9f23-74f2ef65734c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:41.918400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:41.985089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 55/109] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 17:04:41.991699 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bd380-cb1d-4b88-9115-b1066b8ffccc None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:42.092001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:42.117403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bd380-cb1d-4b88-9115-b1066b8ffccc None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:42.161726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2654d-3f30-4587-accf-af4514d5c86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:42.162380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2654d-3f30-4587-accf-af4514d5c86a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:42.162675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2654d-3f30-4587-accf-af4514d5c86a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:42.162958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2654d-3f30-4587-accf-af4514d5c86a None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:42.167187 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2654d-3f30-4587-accf-af4514d5c86a None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:42.168663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2654d-3f30-4587-accf-af4514d5c86a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:42.170588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 55/110] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:41 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:42.177098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794165a-1b61-4937-9f23-74f2ef65734c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:42.232749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2654d-3f30-4587-accf-af4514d5c86a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:42.387571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2654d-3f30-4587-accf-af4514d5c86a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:42.404440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794165a-1b61-4937-9f23-74f2ef65734c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:42.429301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccac26-5ec3-4220-91ae-bdd1376ac16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:42.430239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccac26-5ec3-4220-91ae-bdd1376ac16d None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:42.430748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccac26-5ec3-4220-91ae-bdd1376ac16d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:42.431202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccac26-5ec3-4220-91ae-bdd1376ac16d None admin] PATH_INFO: `/v3/roles/member` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:42.449605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccac26-5ec3-4220-91ae-bdd1376ac16d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-f1ccac26-5ec3-4220-91ae-bdd1376ac16d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:42.455037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:42.455927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:42.456717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:42.459084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 56/111] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:42.460774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767d22d-5467-4a2c-b06d-6480105457f7 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:42.461217 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767d22d-5467-4a2c-b06d-6480105457f7 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:42.461638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767d22d-5467-4a2c-b06d-6480105457f7 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:42.464659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 57/112] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:42.465976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767d22d-5467-4a2c-b06d-6480105457f7 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:42.474862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:42.487576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff3332-51aa-4284-975e-e933570ccd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:42.493878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff3332-51aa-4284-975e-e933570ccd74 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:42.493878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff3332-51aa-4284-975e-e933570ccd74 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:42.493878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff3332-51aa-4284-975e-e933570ccd74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:42.493878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff3332-51aa-4284-975e-e933570ccd74 None admin] RBAC: Adding query filter params (role.id=ef17b1e922a942a083c0f1905c8c523c, scope.project.id=c84516b7267f4b43bf816381e3e60ae6, user.id=f766fe03e07644c5b1f918e54a832d4d) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:42.493878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff3332-51aa-4284-975e-e933570ccd74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:42.493878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff3332-51aa-4284-975e-e933570ccd74 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:42.502110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 56/113] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:42 2022] GET /identity/v3/role_assignments?user.id=f766fe03e07644c5b1f918e54a832d4d&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=ef17b1e922a942a083c0f1905c8c523c => generated 259 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:42.503745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c38c6-dbe7-4943-bb5a-9419f3457a0a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:42.504201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c38c6-dbe7-4943-bb5a-9419f3457a0a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:42.504623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c38c6-dbe7-4943-bb5a-9419f3457a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:42.534789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-852c38c6-dbe7-4943-bb5a-9419f3457a0a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:42.536045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-852c38c6-dbe7-4943-bb5a-9419f3457a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:42.561268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:42.631403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 57/114] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:42.633041 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe5112-75df-4650-b226-a3c32391da6b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:42.633489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe5112-75df-4650-b226-a3c32391da6b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:42.633917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe5112-75df-4650-b226-a3c32391da6b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:42.635512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 58/115] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:42.641879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba0d72-5571-4d1b-80c7-9083857658af None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:42.642369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba0d72-5571-4d1b-80c7-9083857658af None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:42.642952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba0d72-5571-4d1b-80c7-9083857658af None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:42.644507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 59/116] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:42.645941 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0e0d8-918e-465e-8f05-3787f2634927 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:42.646430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0e0d8-918e-465e-8f05-3787f2634927 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:42.646669 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767d22d-5467-4a2c-b06d-6480105457f7 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:42.647032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0e0d8-918e-465e-8f05-3787f2634927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:42.668481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-02c0e0d8-918e-465e-8f05-3787f2634927 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:42.669936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-02c0e0d8-918e-465e-8f05-3787f2634927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:42.727620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9587d-90ea-454e-9572-7b638307eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:42.728321 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9587d-90ea-454e-9572-7b638307eba9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:42.728577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9587d-90ea-454e-9572-7b638307eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:42.729009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9587d-90ea-454e-9572-7b638307eba9 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:42.729887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9587d-90ea-454e-9572-7b638307eba9 None admin] RBAC: Adding query filter params (name=member) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:42.730349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9587d-90ea-454e-9572-7b638307eba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:42.732019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9587d-90ea-454e-9572-7b638307eba9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:42.750105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 60/117] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:42.751433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c109713-20ca-4a11-a3f2-6aacc14875a0 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:42.751724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c109713-20ca-4a11-a3f2-6aacc14875a0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:42.752005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c109713-20ca-4a11-a3f2-6aacc14875a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:42.753268 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 58/118] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:42.758763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9587d-90ea-454e-9572-7b638307eba9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:42.788295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1c109713-20ca-4a11-a3f2-6aacc14875a0 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:42.789704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1c109713-20ca-4a11-a3f2-6aacc14875a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:42.908306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:42.909471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:42.914836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 61/119] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:42.971613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9587d-90ea-454e-9572-7b638307eba9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:43.019249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1ac1c-7647-4859-bdeb-58366bee6e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:43.019903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1ac1c-7647-4859-bdeb-58366bee6e75 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:43.020188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1ac1c-7647-4859-bdeb-58366bee6e75 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:43.020467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1ac1c-7647-4859-bdeb-58366bee6e75 None admin] PATH_INFO: `/v3/users/734f29275e5547fcb788acba34124924` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:43.021966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1ac1c-7647-4859-bdeb-58366bee6e75 None admin] RBAC: Authorizing `identity:get_user(user_id=734f29275e5547fcb788acba34124924)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:43.023472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1ac1c-7647-4859-bdeb-58366bee6e75 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:43.025117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 59/120] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:42 2022] GET /identity/v3/users/734f29275e5547fcb788acba34124924 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:43.030647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c109713-20ca-4a11-a3f2-6aacc14875a0 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:43.146537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c109713-20ca-4a11-a3f2-6aacc14875a0 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:43.183720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508ff9e-cd81-4ce1-b400-8488db738f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:43.184320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508ff9e-cd81-4ce1-b400-8488db738f60 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:43.184562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508ff9e-cd81-4ce1-b400-8488db738f60 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:43.184843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508ff9e-cd81-4ce1-b400-8488db738f60 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:43.186001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508ff9e-cd81-4ce1-b400-8488db738f60 None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:43.187667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508ff9e-cd81-4ce1-b400-8488db738f60 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:43.189461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 62/121] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:43 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:43.195541 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1ac1c-7647-4859-bdeb-58366bee6e75 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:43.207210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508ff9e-cd81-4ce1-b400-8488db738f60 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:43.286271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:43.322200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508ff9e-cd81-4ce1-b400-8488db738f60 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:43.367863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcfac30-d153-434c-8abb-2ee93dfa87ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:43.369538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfac30-d153-434c-8abb-2ee93dfa87ab None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:43.370557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfac30-d153-434c-8abb-2ee93dfa87ab None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:43.372079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfac30-d153-434c-8abb-2ee93dfa87ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:43.391461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcfac30-d153-434c-8abb-2ee93dfa87ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-6dcfac30-d153-434c-8abb-2ee93dfa87ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:43.397308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:43.398360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:04:43.399351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:43.405597 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1ac1c-7647-4859-bdeb-58366bee6e75 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:43.408983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 63/122] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:04:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:43.414537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcfac30-d153-434c-8abb-2ee93dfa87ab None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:43.443488 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:43.446518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77e52f-485b-45d7-81f8-22cc55377739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:43.447441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77e52f-485b-45d7-81f8-22cc55377739 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:43.447887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77e52f-485b-45d7-81f8-22cc55377739 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:43.448303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77e52f-485b-45d7-81f8-22cc55377739 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6/users/734f29275e5547fcb788acba34124924/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:43.450769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77e52f-485b-45d7-81f8-22cc55377739 None admin] RBAC: Authorizing `identity:create_grant(project_id=c84516b7267f4b43bf816381e3e60ae6, user_id=734f29275e5547fcb788acba34124924, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:43.452651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77e52f-485b-45d7-81f8-22cc55377739 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:43.551631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcfac30-d153-434c-8abb-2ee93dfa87ab None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:43.560026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 60/123] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:04:43 2022] PUT /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6/users/734f29275e5547fcb788acba34124924/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:04:43.567323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77e52f-485b-45d7-81f8-22cc55377739 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:43.607730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:43.617149 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b480d0-a3c7-49b4-a40a-77498201784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:43.617807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b480d0-a3c7-49b4-a40a-77498201784c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:43.618147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b480d0-a3c7-49b4-a40a-77498201784c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:43.618428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b480d0-a3c7-49b4-a40a-77498201784c None admin] PATH_INFO: `/v3/roles/member` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:43.632674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b480d0-a3c7-49b4-a40a-77498201784c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-24b480d0-a3c7-49b4-a40a-77498201784c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:43.638403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:43.639375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:43.640177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 64/124] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:43.640822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b480d0-a3c7-49b4-a40a-77498201784c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:43.803299 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77e52f-485b-45d7-81f8-22cc55377739 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:43.817954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b480d0-a3c7-49b4-a40a-77498201784c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:43.856329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d5084-561d-48b5-a3c3-7301be3f1306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:43.857329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d5084-561d-48b5-a3c3-7301be3f1306 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:43.857811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d5084-561d-48b5-a3c3-7301be3f1306 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:43.858279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d5084-561d-48b5-a3c3-7301be3f1306 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:43.859319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d5084-561d-48b5-a3c3-7301be3f1306 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:43.859874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d5084-561d-48b5-a3c3-7301be3f1306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:43.861661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d5084-561d-48b5-a3c3-7301be3f1306 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:43.873352 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 61/125] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:04:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:43.874383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ee835-bfa7-44cb-ba0d-5faa23e3ce34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:43.874930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d5084-561d-48b5-a3c3-7301be3f1306 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:43.875101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ee835-bfa7-44cb-ba0d-5faa23e3ce34 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:43.875349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ee835-bfa7-44cb-ba0d-5faa23e3ce34 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:43.875618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ee835-bfa7-44cb-ba0d-5faa23e3ce34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:43.903867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ee835-bfa7-44cb-ba0d-5faa23e3ce34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-b86ee835-bfa7-44cb-ba0d-5faa23e3ce34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:43.909030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:43.909969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:43.910795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 65/126] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:04:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:43.911459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ee835-bfa7-44cb-ba0d-5faa23e3ce34 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:43.965202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:44.013810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d5084-561d-48b5-a3c3-7301be3f1306 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:44.050740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17094f51-337b-41fc-8e9f-91d77913db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:44.051347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17094f51-337b-41fc-8e9f-91d77913db75 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.051623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17094f51-337b-41fc-8e9f-91d77913db75 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.051892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17094f51-337b-41fc-8e9f-91d77913db75 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.052840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17094f51-337b-41fc-8e9f-91d77913db75 None admin] RBAC: Adding query filter params (name=member) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:44.053178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17094f51-337b-41fc-8e9f-91d77913db75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:44.054724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17094f51-337b-41fc-8e9f-91d77913db75 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:44.056765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:44.063272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 62/127] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:44.064103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17094f51-337b-41fc-8e9f-91d77913db75 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:44.144545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ee835-bfa7-44cb-ba0d-5faa23e3ce34 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:44.179415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17094f51-337b-41fc-8e9f-91d77913db75 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:44.207388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:44.210553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc32c7-7c21-4b70-8778-562416687f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:44.211686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc32c7-7c21-4b70-8778-562416687f24 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.212109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc32c7-7c21-4b70-8778-562416687f24 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.212521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc32c7-7c21-4b70-8778-562416687f24 None admin] PATH_INFO: `/v3/users/734f29275e5547fcb788acba34124924` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.214011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc32c7-7c21-4b70-8778-562416687f24 None admin] RBAC: Authorizing `identity:get_user(user_id=734f29275e5547fcb788acba34124924)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:44.218598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc32c7-7c21-4b70-8778-562416687f24 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:44.218598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 66/128] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:43 2022] GET /identity/v3/users/734f29275e5547fcb788acba34124924 => generated 275 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:44.218598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc32c7-7c21-4b70-8778-562416687f24 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:44.230033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef8336-9d53-4a84-87d8-6011e7682939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:44.230680 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef8336-9d53-4a84-87d8-6011e7682939 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.231134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef8336-9d53-4a84-87d8-6011e7682939 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.231541 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef8336-9d53-4a84-87d8-6011e7682939 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.232471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef8336-9d53-4a84-87d8-6011e7682939 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:44.232909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef8336-9d53-4a84-87d8-6011e7682939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:44.234614 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef8336-9d53-4a84-87d8-6011e7682939 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:44.245368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 63/129] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:44.250512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5363-d864-470a-8710-17d6f07b7bd0 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.251084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5363-d864-470a-8710-17d6f07b7bd0 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.251618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5363-d864-470a-8710-17d6f07b7bd0 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.253032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 64/130] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:44.253999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab5363-d864-470a-8710-17d6f07b7bd0 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:44.342059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc32c7-7c21-4b70-8778-562416687f24 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:44.361107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:44.376558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab5363-d864-470a-8710-17d6f07b7bd0 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:44.381963 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed507130-f622-475e-bd19-b5367fae0cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:44.382924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed507130-f622-475e-bd19-b5367fae0cb2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.383362 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed507130-f622-475e-bd19-b5367fae0cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.383777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed507130-f622-475e-bd19-b5367fae0cb2 None admin] PATH_INFO: `/v3/users/734f29275e5547fcb788acba34124924` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.385259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed507130-f622-475e-bd19-b5367fae0cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=734f29275e5547fcb788acba34124924)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:44.386999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed507130-f622-475e-bd19-b5367fae0cb2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:44.393335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 67/131] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:44 2022] GET /identity/v3/users/734f29275e5547fcb788acba34124924 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:44.393335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed507130-f622-475e-bd19-b5367fae0cb2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:44.421406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dbd25-8176-4c6b-a534-bdb71062c039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:44.422379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dbd25-8176-4c6b-a534-bdb71062c039 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.422967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dbd25-8176-4c6b-a534-bdb71062c039 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.423460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dbd25-8176-4c6b-a534-bdb71062c039 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.425087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dbd25-8176-4c6b-a534-bdb71062c039 None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:44.426875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dbd25-8176-4c6b-a534-bdb71062c039 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:44.434841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 65/132] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:44 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:44.436401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67ae0-14ae-4899-ad85-57951306ad78 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.437017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67ae0-14ae-4899-ad85-57951306ad78 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.437480 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67ae0-14ae-4899-ad85-57951306ad78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.456482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-5ff67ae0-14ae-4899-ad85-57951306ad78 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:44.457977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5ff67ae0-14ae-4899-ad85-57951306ad78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:44.504908 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:44.513365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed507130-f622-475e-bd19-b5367fae0cb2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:44.531241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 66/133] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:44.532434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67ae0-14ae-4899-ad85-57951306ad78 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:44.570958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e94dfd-a54e-4859-b68e-a89d1def9ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:44.571609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e94dfd-a54e-4859-b68e-a89d1def9ac9 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.571951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e94dfd-a54e-4859-b68e-a89d1def9ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.572233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e94dfd-a54e-4859-b68e-a89d1def9ac9 None admin] PATH_INFO: `/v3/users/b8814a79ac534e6e8b81d70404045912` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.576474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e94dfd-a54e-4859-b68e-a89d1def9ac9 None admin] RBAC: Authorizing `identity:get_user(user_id=b8814a79ac534e6e8b81d70404045912)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:44.577902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e94dfd-a54e-4859-b68e-a89d1def9ac9 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:44.579893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 68/134] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:44 2022] GET /identity/v3/users/b8814a79ac534e6e8b81d70404045912 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:44.586059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e94dfd-a54e-4859-b68e-a89d1def9ac9 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:44.658879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:44.682430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67ae0-14ae-4899-ad85-57951306ad78 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:44.709005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e94dfd-a54e-4859-b68e-a89d1def9ac9 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:44.750325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcb080-e5c2-4d89-95c9-50e50c7b6a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:44.751426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcb080-e5c2-4d89-95c9-50e50c7b6a25 None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.751963 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcb080-e5c2-4d89-95c9-50e50c7b6a25 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.752434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcb080-e5c2-4d89-95c9-50e50c7b6a25 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6/users/734f29275e5547fcb788acba34124924/roles/c2fe89b1b9fa47b19e02efd64ace9000` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.754928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcb080-e5c2-4d89-95c9-50e50c7b6a25 None admin] RBAC: Authorizing `identity:create_grant(project_id=c84516b7267f4b43bf816381e3e60ae6, user_id=734f29275e5547fcb788acba34124924, role_id=c2fe89b1b9fa47b19e02efd64ace9000)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:44.757162 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcb080-e5c2-4d89-95c9-50e50c7b6a25 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:44.762971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c363ba0-fed7-4e5c-b83d-26e2e9233154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:44.763697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c363ba0-fed7-4e5c-b83d-26e2e9233154 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.763987 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c363ba0-fed7-4e5c-b83d-26e2e9233154 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.764288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c363ba0-fed7-4e5c-b83d-26e2e9233154 None admin] PATH_INFO: `/v3/projects/3cd1942bafc649eaa8bbd93187a3a618` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.767257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c363ba0-fed7-4e5c-b83d-26e2e9233154 None admin] RBAC: Authorizing `identity:get_project(project_id=3cd1942bafc649eaa8bbd93187a3a618)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:44.771362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c363ba0-fed7-4e5c-b83d-26e2e9233154 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:44.772207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 67/135] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:44 2022] GET /identity/v3/projects/3cd1942bafc649eaa8bbd93187a3a618 => generated 312 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:44.773326 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c363ba0-fed7-4e5c-b83d-26e2e9233154 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:44.832657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:44.857471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 69/136] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:04:44 2022] PUT /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6/users/734f29275e5547fcb788acba34124924/roles/c2fe89b1b9fa47b19e02efd64ace9000 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:04:44.858307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcb080-e5c2-4d89-95c9-50e50c7b6a25 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:44.915025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c363ba0-fed7-4e5c-b83d-26e2e9233154 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:44.953504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb6280-101a-4736-8853-83da26aadb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:44.954150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6280-101a-4736-8853-83da26aadb2a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.954434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6280-101a-4736-8853-83da26aadb2a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.954740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6280-101a-4736-8853-83da26aadb2a None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.955958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb6280-101a-4736-8853-83da26aadb2a None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:44.957432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb6280-101a-4736-8853-83da26aadb2a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:44.959887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 68/137] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:44 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:44.960852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f7fa-cb29-4cd6-b391-eea5e637e7c0 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:44.961131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f7fa-cb29-4cd6-b391-eea5e637e7c0 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:44.961399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f7fa-cb29-4cd6-b391-eea5e637e7c0 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:44.967700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 69/138] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:44.968563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8f7fa-cb29-4cd6-b391-eea5e637e7c0 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:44.992192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcb080-e5c2-4d89-95c9-50e50c7b6a25 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:45.008979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:45.034834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dae5cd-b31f-4f79-b1d9-be86f61afb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:45.035635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dae5cd-b31f-4f79-b1d9-be86f61afb3d None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:45.035925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dae5cd-b31f-4f79-b1d9-be86f61afb3d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:45.036346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dae5cd-b31f-4f79-b1d9-be86f61afb3d None admin] PATH_INFO: `/v3/projects/3cd1942bafc649eaa8bbd93187a3a618/users/734f29275e5547fcb788acba34124924/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:45.038780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dae5cd-b31f-4f79-b1d9-be86f61afb3d None admin] RBAC: Authorizing `identity:create_grant(project_id=3cd1942bafc649eaa8bbd93187a3a618, user_id=734f29275e5547fcb788acba34124924, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:45.041964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dae5cd-b31f-4f79-b1d9-be86f61afb3d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:45.091510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8f7fa-cb29-4cd6-b391-eea5e637e7c0 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:45.103487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 70/139] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:04:44 2022] PUT /identity/v3/projects/3cd1942bafc649eaa8bbd93187a3a618/users/734f29275e5547fcb788acba34124924/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:04:45.104773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a744a-33e1-42b2-b866-48b03c442232 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:45.105040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a744a-33e1-42b2-b866-48b03c442232 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:45.105319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a744a-33e1-42b2-b866-48b03c442232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:45.109786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:45.124799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-059a744a-33e1-42b2-b866-48b03c442232 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:45.126134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-059a744a-33e1-42b2-b866-48b03c442232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:45.140674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0f9ca-443f-4c93-b770-2bc72792c34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:45.141891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0f9ca-443f-4c93-b770-2bc72792c34b None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:45.142302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0f9ca-443f-4c93-b770-2bc72792c34b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:45.142664 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0f9ca-443f-4c93-b770-2bc72792c34b None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6/users/b8814a79ac534e6e8b81d70404045912/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:45.144798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0f9ca-443f-4c93-b770-2bc72792c34b None admin] RBAC: Authorizing `identity:create_grant(project_id=c84516b7267f4b43bf816381e3e60ae6, user_id=b8814a79ac534e6e8b81d70404045912, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:45.147991 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0f9ca-443f-4c93-b770-2bc72792c34b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:45.211182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 71/140] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:45.211980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a744a-33e1-42b2-b866-48b03c442232 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:45.217722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 70/141] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:04:44 2022] PUT /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6/users/b8814a79ac534e6e8b81d70404045912/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:04:45.363903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a744a-33e1-42b2-b866-48b03c442232 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:45.367310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:45.403266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63544f-3798-4edb-8532-ddb6424902c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:45.403936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63544f-3798-4edb-8532-ddb6424902c3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:45.404214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63544f-3798-4edb-8532-ddb6424902c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:45.404488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63544f-3798-4edb-8532-ddb6424902c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:45.405774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63544f-3798-4edb-8532-ddb6424902c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:45.407322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63544f-3798-4edb-8532-ddb6424902c3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:45.409351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 72/142] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:45.415809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0f9ca-443f-4c93-b770-2bc72792c34b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:45.533437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0f9ca-443f-4c93-b770-2bc72792c34b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:45.581828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e629de-8377-4f88-b570-3fe8baee1244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:45.582584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e629de-8377-4f88-b570-3fe8baee1244 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:45.582865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e629de-8377-4f88-b570-3fe8baee1244 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:45.583140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e629de-8377-4f88-b570-3fe8baee1244 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:45.584576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e629de-8377-4f88-b570-3fe8baee1244 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:45.586369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e629de-8377-4f88-b570-3fe8baee1244 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:45.614998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63544f-3798-4edb-8532-ddb6424902c3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:45.701471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 71/143] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:04:45 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:04:45.748005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63544f-3798-4edb-8532-ddb6424902c3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:45.793773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf689903-56bc-474f-b6fe-4454577400f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:45.794705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf689903-56bc-474f-b6fe-4454577400f0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:45.795129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf689903-56bc-474f-b6fe-4454577400f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:45.795544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf689903-56bc-474f-b6fe-4454577400f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:45.816785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf689903-56bc-474f-b6fe-4454577400f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-bf689903-56bc-474f-b6fe-4454577400f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:45.822776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:45.823849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:04:45.824802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:45.831279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 73/144] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:04:45.837965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e629de-8377-4f88-b570-3fe8baee1244 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:45.893926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:45.984999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e629de-8377-4f88-b570-3fe8baee1244 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:46.002705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b568f0-d7ed-4877-80fc-9fdb46559965 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:46.003331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b568f0-d7ed-4877-80fc-9fdb46559965 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:46.003806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b568f0-d7ed-4877-80fc-9fdb46559965 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:46.006256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 74/145] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:46.012772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f51ee5-1a5e-4dc5-9b82-e22f449960a4 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:46.013357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f51ee5-1a5e-4dc5-9b82-e22f449960a4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:46.013858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f51ee5-1a5e-4dc5-9b82-e22f449960a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:46.025102 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87606637-fb6e-4504-a561-d1558ed0b5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:46.025729 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87606637-fb6e-4504-a561-d1558ed0b5a4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:46.026031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87606637-fb6e-4504-a561-d1558ed0b5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:46.026278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87606637-fb6e-4504-a561-d1558ed0b5a4 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:46.027019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87606637-fb6e-4504-a561-d1558ed0b5a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:46.027325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87606637-fb6e-4504-a561-d1558ed0b5a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:46.034520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87606637-fb6e-4504-a561-d1558ed0b5a4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:46.042682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 72/146] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:46.048310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87606637-fb6e-4504-a561-d1558ed0b5a4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:46.053881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f7f51ee5-1a5e-4dc5-9b82-e22f449960a4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:46.055218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f7f51ee5-1a5e-4dc5-9b82-e22f449960a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:46.088001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:46.169805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 75/147] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:46.191019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87606637-fb6e-4504-a561-d1558ed0b5a4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:46.194820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:46.238304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ea07c-f963-4fda-b2cd-5e654a5f16c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:46.239074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ea07c-f963-4fda-b2cd-5e654a5f16c7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:46.239259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ea07c-f963-4fda-b2cd-5e654a5f16c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:46.240062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ea07c-f963-4fda-b2cd-5e654a5f16c7 None admin] PATH_INFO: `/v3/users/f766fe03e07644c5b1f918e54a832d4d` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:46.241236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ea07c-f963-4fda-b2cd-5e654a5f16c7 None admin] RBAC: Authorizing `identity:get_user(user_id=f766fe03e07644c5b1f918e54a832d4d)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:46.242720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ea07c-f963-4fda-b2cd-5e654a5f16c7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:46.244327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 73/148] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:46 2022] GET /identity/v3/users/f766fe03e07644c5b1f918e54a832d4d => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:46.250004 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f51ee5-1a5e-4dc5-9b82-e22f449960a4 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:46.378828 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f51ee5-1a5e-4dc5-9b82-e22f449960a4 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:46.424043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d3cf2-5450-46c1-b276-4f3ee43eec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:46.424759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d3cf2-5450-46c1-b276-4f3ee43eec54 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:46.425039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d3cf2-5450-46c1-b276-4f3ee43eec54 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:46.425314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d3cf2-5450-46c1-b276-4f3ee43eec54 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:46.426540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d3cf2-5450-46c1-b276-4f3ee43eec54 None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:46.428666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d3cf2-5450-46c1-b276-4f3ee43eec54 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:46.430738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 76/149] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:46 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:46.436392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ea07c-f963-4fda-b2cd-5e654a5f16c7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:46.522496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d3cf2-5450-46c1-b276-4f3ee43eec54 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:46.560639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:46.608763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ea07c-f963-4fda-b2cd-5e654a5f16c7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:46.651321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d3cf2-5450-46c1-b276-4f3ee43eec54 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:46.691095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab276f-be2b-4551-b3ff-a494fe70e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:46.691740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab276f-be2b-4551-b3ff-a494fe70e5fd None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:46.692020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab276f-be2b-4551-b3ff-a494fe70e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:46.692292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab276f-be2b-4551-b3ff-a494fe70e5fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:46.693657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de567c-1085-4fdf-a062-4395b662a85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:46.694550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de567c-1085-4fdf-a062-4395b662a85d None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:46.698865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de567c-1085-4fdf-a062-4395b662a85d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:46.699147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de567c-1085-4fdf-a062-4395b662a85d None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6/users/f766fe03e07644c5b1f918e54a832d4d/roles/ef17b1e922a942a083c0f1905c8c523c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:46.701357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de567c-1085-4fdf-a062-4395b662a85d None admin] RBAC: Authorizing `identity:create_grant(project_id=c84516b7267f4b43bf816381e3e60ae6, user_id=f766fe03e07644c5b1f918e54a832d4d, role_id=ef17b1e922a942a083c0f1905c8c523c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:46.702846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de567c-1085-4fdf-a062-4395b662a85d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:46.704731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab276f-be2b-4551-b3ff-a494fe70e5fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-42ab276f-be2b-4551-b3ff-a494fe70e5fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:46.710144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:46.711050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:46.711832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 77/150] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:46.717196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab276f-be2b-4551-b3ff-a494fe70e5fd None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:46.800980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 74/151] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:04:46 2022] PUT /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6/users/f766fe03e07644c5b1f918e54a832d4d/roles/ef17b1e922a942a083c0f1905c8c523c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:04:46.821110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:46.840526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab276f-be2b-4551-b3ff-a494fe70e5fd None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:46.885453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57630ae5-3d72-4ab2-897e-214f25020399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:46.886087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57630ae5-3d72-4ab2-897e-214f25020399 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:46.886367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57630ae5-3d72-4ab2-897e-214f25020399 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:46.886670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57630ae5-3d72-4ab2-897e-214f25020399 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:46.887358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57630ae5-3d72-4ab2-897e-214f25020399 None admin] RBAC: Adding query filter params (name=member) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:46.887660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57630ae5-3d72-4ab2-897e-214f25020399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:46.889078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57630ae5-3d72-4ab2-897e-214f25020399 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:46.903997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 78/152] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:46.909691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de567c-1085-4fdf-a062-4395b662a85d None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:46.948689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:47.064302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de567c-1085-4fdf-a062-4395b662a85d None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:47.104979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63893ac-f18b-4ec6-8c41-5228ebd97286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:47.105621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63893ac-f18b-4ec6-8c41-5228ebd97286 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:47.105937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63893ac-f18b-4ec6-8c41-5228ebd97286 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:47.106284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63893ac-f18b-4ec6-8c41-5228ebd97286 None admin] PATH_INFO: `/v3/users/734f29275e5547fcb788acba34124924` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:47.111760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63893ac-f18b-4ec6-8c41-5228ebd97286 None admin] RBAC: Authorizing `identity:get_user(user_id=734f29275e5547fcb788acba34124924)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:47.113278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63893ac-f18b-4ec6-8c41-5228ebd97286 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:47.116136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 75/153] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:46 2022] GET /identity/v3/users/734f29275e5547fcb788acba34124924 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:47.121746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57630ae5-3d72-4ab2-897e-214f25020399 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:47.274896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57630ae5-3d72-4ab2-897e-214f25020399 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:47.328608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad70ab-f254-479c-9fc3-d83fc39df819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:47.329306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad70ab-f254-479c-9fc3-d83fc39df819 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:47.332191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad70ab-f254-479c-9fc3-d83fc39df819 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:47.332191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad70ab-f254-479c-9fc3-d83fc39df819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:47.358962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:47.420550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad70ab-f254-479c-9fc3-d83fc39df819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:47.422193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad70ab-f254-479c-9fc3-d83fc39df819 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:47.546501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 79/154] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:47.553187 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63893ac-f18b-4ec6-8c41-5228ebd97286 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:47.652512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:47.668813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63893ac-f18b-4ec6-8c41-5228ebd97286 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:47.708547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11445877-d406-4bc4-8732-19a3edaf8644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:47.709232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11445877-d406-4bc4-8732-19a3edaf8644 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:47.709508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11445877-d406-4bc4-8732-19a3edaf8644 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:47.709776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11445877-d406-4bc4-8732-19a3edaf8644 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:47.711009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11445877-d406-4bc4-8732-19a3edaf8644 None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:47.712530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11445877-d406-4bc4-8732-19a3edaf8644 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:47.714291 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 76/155] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:47 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:47.720219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad70ab-f254-479c-9fc3-d83fc39df819 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:47.788631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:47.844908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad70ab-f254-479c-9fc3-d83fc39df819 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:47.896757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00e93c-34e6-40f4-8284-454b262cf691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:47.897378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00e93c-34e6-40f4-8284-454b262cf691 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:47.897654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00e93c-34e6-40f4-8284-454b262cf691 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:47.897924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00e93c-34e6-40f4-8284-454b262cf691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:47.898695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00e93c-34e6-40f4-8284-454b262cf691 None admin] RBAC: Adding query filter params (role.id=13accf89c12040508974db47c6905b31, scope.project.id=c84516b7267f4b43bf816381e3e60ae6, user.id=734f29275e5547fcb788acba34124924) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:47.899004 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00e93c-34e6-40f4-8284-454b262cf691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:47.900571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00e93c-34e6-40f4-8284-454b262cf691 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:47.907748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 80/156] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:47 2022] GET /identity/v3/role_assignments?user.id=734f29275e5547fcb788acba34124924&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=13accf89c12040508974db47c6905b31 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:48.348562 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2d293-7491-4976-9089-985631bcc264 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:48.349124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2d293-7491-4976-9089-985631bcc264 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:48.349424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2d293-7491-4976-9089-985631bcc264 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:48.350854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 77/157] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:48.363539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0194be2-c6bd-4a85-aec3-b4f52f23b224 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:48.364055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0194be2-c6bd-4a85-aec3-b4f52f23b224 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:48.364055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0194be2-c6bd-4a85-aec3-b4f52f23b224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:48.384426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f9f0d-a606-411f-93cb-2e93ab6c49b3 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:48.384719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f9f0d-a606-411f-93cb-2e93ab6c49b3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:48.384972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f9f0d-a606-411f-93cb-2e93ab6c49b3 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:48.386328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 78/158] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:04:48.391870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e7686-0d46-4429-ad37-2df81b6481cb None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:48.392155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e7686-0d46-4429-ad37-2df81b6481cb None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:48.392441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e7686-0d46-4429-ad37-2df81b6481cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:48.395646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f0194be2-c6bd-4a85-aec3-b4f52f23b224 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:48.397366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f0194be2-c6bd-4a85-aec3-b4f52f23b224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:48.426641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-6a1e7686-0d46-4429-ad37-2df81b6481cb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:48.428441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-6a1e7686-0d46-4429-ad37-2df81b6481cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:48.544192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 81/159] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:04:48.545573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce14aa-e951-4160-b00e-17816c787dd0 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:48.545859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce14aa-e951-4160-b00e-17816c787dd0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:48.546136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce14aa-e951-4160-b00e-17816c787dd0 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:48.547542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 82/160] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:48.556069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32767876-a98f-4ff6-8829-3e0490786bbb None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:48.556343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32767876-a98f-4ff6-8829-3e0490786bbb None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:48.556777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32767876-a98f-4ff6-8829-3e0490786bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:48.568405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 79/161] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:48.591507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-32767876-a98f-4ff6-8829-3e0490786bbb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:48.592929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-32767876-a98f-4ff6-8829-3e0490786bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:48.654566 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:48.692621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 83/162] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:04:49.020488 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e7686-0d46-4429-ad37-2df81b6481cb None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:49.117487 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:49.134512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32767876-a98f-4ff6-8829-3e0490786bbb None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:49.138910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e7686-0d46-4429-ad37-2df81b6481cb None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:49.221400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006eea94-f7f4-488c-a857-8aed79b6ee78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:49.222046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eea94-f7f4-488c-a857-8aed79b6ee78 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:49.222352 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eea94-f7f4-488c-a857-8aed79b6ee78 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:49.222648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eea94-f7f4-488c-a857-8aed79b6ee78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:49.240703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006eea94-f7f4-488c-a857-8aed79b6ee78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-006eea94-f7f4-488c-a857-8aed79b6ee78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:49.246163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:49.247124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:49.247893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 80/163] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:04:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:49.260057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006eea94-f7f4-488c-a857-8aed79b6ee78 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:49.362094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32767876-a98f-4ff6-8829-3e0490786bbb None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:49.460810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006eea94-f7f4-488c-a857-8aed79b6ee78 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:49.519951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a906cc8-82a7-4e38-a9c1-71de9b688e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:49.521197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a906cc8-82a7-4e38-a9c1-71de9b688e4b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:49.521515 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a906cc8-82a7-4e38-a9c1-71de9b688e4b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:49.521860 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a906cc8-82a7-4e38-a9c1-71de9b688e4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:49.524387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c45bc0-9b77-4bf6-aa9a-5f427698955c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:49.524997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c45bc0-9b77-4bf6-aa9a-5f427698955c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:49.525497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c45bc0-9b77-4bf6-aa9a-5f427698955c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:49.525769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c45bc0-9b77-4bf6-aa9a-5f427698955c None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:49.526441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c45bc0-9b77-4bf6-aa9a-5f427698955c None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:49.526764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c45bc0-9b77-4bf6-aa9a-5f427698955c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:49.528223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c45bc0-9b77-4bf6-aa9a-5f427698955c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:49.548127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 81/164] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:49.549278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c45bc0-9b77-4bf6-aa9a-5f427698955c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:49.558738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a906cc8-82a7-4e38-a9c1-71de9b688e4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-9a906cc8-82a7-4e38-a9c1-71de9b688e4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:49.563982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:49.564920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:49.566126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 84/165] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:04:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:04:49.575466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a906cc8-82a7-4e38-a9c1-71de9b688e4b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:49.578869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:49.653869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:49.741782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c45bc0-9b77-4bf6-aa9a-5f427698955c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:49.763276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a906cc8-82a7-4e38-a9c1-71de9b688e4b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:49.800328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bfdf5-6c97-4c5c-bf46-7c2cbc2599e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:49.800527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99639e37-8488-48ba-8d60-880ec1632dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:49.800958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bfdf5-6c97-4c5c-bf46-7c2cbc2599e9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:49.801064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99639e37-8488-48ba-8d60-880ec1632dc9 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:49.801221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bfdf5-6c97-4c5c-bf46-7c2cbc2599e9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:49.801401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99639e37-8488-48ba-8d60-880ec1632dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:49.801490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bfdf5-6c97-4c5c-bf46-7c2cbc2599e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:49.801642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99639e37-8488-48ba-8d60-880ec1632dc9 None admin] PATH_INFO: `/v3/users/b8814a79ac534e6e8b81d70404045912` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:49.807359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99639e37-8488-48ba-8d60-880ec1632dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=b8814a79ac534e6e8b81d70404045912)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:49.808830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99639e37-8488-48ba-8d60-880ec1632dc9 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:49.812827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 85/166] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:49 2022] GET /identity/v3/users/b8814a79ac534e6e8b81d70404045912 => generated 247 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:49.813622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99639e37-8488-48ba-8d60-880ec1632dc9 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:49.834863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09bfdf5-6c97-4c5c-bf46-7c2cbc2599e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-b09bfdf5-6c97-4c5c-bf46-7c2cbc2599e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:49.840642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:49.841510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:49.842742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 82/167] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:49.843464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4b817-07fc-4793-8e6f-dd5001481cd9 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:49.843464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4b817-07fc-4793-8e6f-dd5001481cd9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:49.843841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4b817-07fc-4793-8e6f-dd5001481cd9 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:49.844914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 83/168] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:49.846387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4b817-07fc-4793-8e6f-dd5001481cd9 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:49.978745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:50.019647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99639e37-8488-48ba-8d60-880ec1632dc9 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:50.023352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:50.033818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4b817-07fc-4793-8e6f-dd5001481cd9 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:50.091391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b641dd8-ca82-44c2-80f5-19331c64a5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:50.092809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b641dd8-ca82-44c2-80f5-19331c64a5df None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.093818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b641dd8-ca82-44c2-80f5-19331c64a5df None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.094777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b641dd8-ca82-44c2-80f5-19331c64a5df None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.096440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b641dd8-ca82-44c2-80f5-19331c64a5df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:50.097435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b641dd8-ca82-44c2-80f5-19331c64a5df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:50.098231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b5c08-7174-40ae-b972-f1d73b0a58b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:50.099850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b5c08-7174-40ae-b972-f1d73b0a58b6 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.101089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b5c08-7174-40ae-b972-f1d73b0a58b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.101690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b641dd8-ca82-44c2-80f5-19331c64a5df None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:50.102154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b5c08-7174-40ae-b972-f1d73b0a58b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.119344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 86/169] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:04:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:50.120613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b641dd8-ca82-44c2-80f5-19331c64a5df None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:50.231043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b5c08-7174-40ae-b972-f1d73b0a58b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:50.233712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b5c08-7174-40ae-b972-f1d73b0a58b6 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:50.304054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b641dd8-ca82-44c2-80f5-19331c64a5df None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:50.363426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:50.380489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0ddc6-98a1-4fec-ad32-14d089c871d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:50.381591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0ddc6-98a1-4fec-ad32-14d089c871d3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.381700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0ddc6-98a1-4fec-ad32-14d089c871d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.381891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0ddc6-98a1-4fec-ad32-14d089c871d3 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.382604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0ddc6-98a1-4fec-ad32-14d089c871d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:50.382906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0ddc6-98a1-4fec-ad32-14d089c871d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:50.384291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0ddc6-98a1-4fec-ad32-14d089c871d3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:50.408835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 84/170] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:50.412867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 87/171] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:50.412867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0ddc6-98a1-4fec-ad32-14d089c871d3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:50.414442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aa1ad-5288-44ff-b530-2df91c7e8f54 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.414954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aa1ad-5288-44ff-b530-2df91c7e8f54 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.415535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aa1ad-5288-44ff-b530-2df91c7e8f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.438069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b69aa1ad-5288-44ff-b530-2df91c7e8f54 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:50.439741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b69aa1ad-5288-44ff-b530-2df91c7e8f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:50.456936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:50.524855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 85/172] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:50.525708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aa1ad-5288-44ff-b530-2df91c7e8f54 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:50.564435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0ddc6-98a1-4fec-ad32-14d089c871d3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:50.614035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ebffd-c36a-4c92-82db-4f14d6220917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:50.615584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ebffd-c36a-4c92-82db-4f14d6220917 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.616177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ebffd-c36a-4c92-82db-4f14d6220917 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.617159 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ebffd-c36a-4c92-82db-4f14d6220917 None admin] PATH_INFO: `/v3/users/734f29275e5547fcb788acba34124924` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.618650 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ebffd-c36a-4c92-82db-4f14d6220917 None admin] RBAC: Authorizing `identity:get_user(user_id=734f29275e5547fcb788acba34124924)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:50.620383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ebffd-c36a-4c92-82db-4f14d6220917 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:50.623760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 88/173] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:50 2022] GET /identity/v3/users/734f29275e5547fcb788acba34124924 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:50.625178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ebffd-c36a-4c92-82db-4f14d6220917 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:50.660092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:50.673178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aa1ad-5288-44ff-b530-2df91c7e8f54 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:50.715848 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882135b-622b-4c5f-8aff-4aba976edffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:50.716855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882135b-622b-4c5f-8aff-4aba976edffb None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.717332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882135b-622b-4c5f-8aff-4aba976edffb None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.717821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882135b-622b-4c5f-8aff-4aba976edffb None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.719164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882135b-622b-4c5f-8aff-4aba976edffb None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:50.720885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882135b-622b-4c5f-8aff-4aba976edffb None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:50.722766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 86/174] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:50 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:50.724670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843b47f-1ac9-4a16-8cd5-a742cf5f5533 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.725199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843b47f-1ac9-4a16-8cd5-a742cf5f5533 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.725649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843b47f-1ac9-4a16-8cd5-a742cf5f5533 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.727162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 87/175] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:50.732137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843b47f-1ac9-4a16-8cd5-a742cf5f5533 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:50.776449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:50.783185 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ebffd-c36a-4c92-82db-4f14d6220917 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:50.821879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8fffd1-4bf6-492c-af5d-9d05d1b969ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:50.822545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fffd1-4bf6-492c-af5d-9d05d1b969ee None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.822823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fffd1-4bf6-492c-af5d-9d05d1b969ee None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.823089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fffd1-4bf6-492c-af5d-9d05d1b969ee None admin] PATH_INFO: `/v3/users/734f29275e5547fcb788acba34124924` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.824223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8fffd1-4bf6-492c-af5d-9d05d1b969ee None admin] RBAC: Authorizing `identity:get_user(user_id=734f29275e5547fcb788acba34124924)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:50.825667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8fffd1-4bf6-492c-af5d-9d05d1b969ee None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:50.827291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 89/176] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:50 2022] GET /identity/v3/users/734f29275e5547fcb788acba34124924 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:50.828449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63853cf2-6500-413c-80a3-4f899f89bcf1 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.828724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63853cf2-6500-413c-80a3-4f899f89bcf1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.828993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63853cf2-6500-413c-80a3-4f899f89bcf1 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.830188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 90/177] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:50.835026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63853cf2-6500-413c-80a3-4f899f89bcf1 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:50.861013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843b47f-1ac9-4a16-8cd5-a742cf5f5533 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:50.900357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03b4da-c3aa-4f20-b845-0b3779cb1922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:50.900960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b4da-c3aa-4f20-b845-0b3779cb1922 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.901230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b4da-c3aa-4f20-b845-0b3779cb1922 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.901495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b4da-c3aa-4f20-b845-0b3779cb1922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.933350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:50.946934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63853cf2-6500-413c-80a3-4f899f89bcf1 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:50.973163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03b4da-c3aa-4f20-b845-0b3779cb1922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:50.975359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03b4da-c3aa-4f20-b845-0b3779cb1922 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:50.981188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f0f11-6d10-493a-a1a4-849d2dd435b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:50.981787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f0f11-6d10-493a-a1a4-849d2dd435b1 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.982060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f0f11-6d10-493a-a1a4-849d2dd435b1 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.982324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f0f11-6d10-493a-a1a4-849d2dd435b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:50.983073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f0f11-6d10-493a-a1a4-849d2dd435b1 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=c84516b7267f4b43bf816381e3e60ae6, user.id=b8814a79ac534e6e8b81d70404045912) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:50.983376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f0f11-6d10-493a-a1a4-849d2dd435b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:50.984846 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f0f11-6d10-493a-a1a4-849d2dd435b1 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:50.992173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 91/178] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:50 2022] GET /identity/v3/role_assignments?user.id=b8814a79ac534e6e8b81d70404045912&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=4b10782d99b74f5bb453b82286012e0c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:50.993438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3ea56-481f-45cb-a452-b7d6e2cfd4c2 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:50.993720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3ea56-481f-45cb-a452-b7d6e2cfd4c2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:50.993995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3ea56-481f-45cb-a452-b7d6e2cfd4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:51.013181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-3cb3ea56-481f-45cb-a452-b7d6e2cfd4c2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:51.014433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3cb3ea56-481f-45cb-a452-b7d6e2cfd4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:51.039764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:51.104526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 92/179] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:51.106291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952b5cb-58d1-47b6-b348-d200719fd040 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:51.106820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952b5cb-58d1-47b6-b348-d200719fd040 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:51.107295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952b5cb-58d1-47b6-b348-d200719fd040 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:51.113502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 93/180] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:51.115451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952b5cb-58d1-47b6-b348-d200719fd040 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:51.119317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 88/181] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:51.120757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522d000-5a2b-4a77-8ff0-69354f3a8902 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:51.121165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522d000-5a2b-4a77-8ff0-69354f3a8902 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:51.121444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522d000-5a2b-4a77-8ff0-69354f3a8902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:51.145402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2522d000-5a2b-4a77-8ff0-69354f3a8902 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:51.147553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2522d000-5a2b-4a77-8ff0-69354f3a8902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:51.232411 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:51.244935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 89/182] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:51.245686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522d000-5a2b-4a77-8ff0-69354f3a8902 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:51.267012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952b5cb-58d1-47b6-b348-d200719fd040 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:51.322055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005429d2-454c-44b6-88f5-30c2ee1a8697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:51.324706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005429d2-454c-44b6-88f5-30c2ee1a8697 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:51.324706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005429d2-454c-44b6-88f5-30c2ee1a8697 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:51.324981 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005429d2-454c-44b6-88f5-30c2ee1a8697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:51.358214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:51.386516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522d000-5a2b-4a77-8ff0-69354f3a8902 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:51.414379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005429d2-454c-44b6-88f5-30c2ee1a8697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:51.415974 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005429d2-454c-44b6-88f5-30c2ee1a8697 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:51.425774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8689c-cccb-4496-a7c0-5d6dd0fe6681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:51.426417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8689c-cccb-4496-a7c0-5d6dd0fe6681 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:51.427252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8689c-cccb-4496-a7c0-5d6dd0fe6681 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:51.427527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8689c-cccb-4496-a7c0-5d6dd0fe6681 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:51.439152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8689c-cccb-4496-a7c0-5d6dd0fe6681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-06e8689c-cccb-4496-a7c0-5d6dd0fe6681 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:51.444567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:51.445777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:51.446682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 90/183] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:04:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:51.447343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1af6-6fcf-4e76-98c9-2289473eade1 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:51.450531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1af6-6fcf-4e76-98c9-2289473eade1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:51.450531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1af6-6fcf-4e76-98c9-2289473eade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:51.468007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ce6f1af6-6fcf-4e76-98c9-2289473eade1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:51.469476 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ce6f1af6-6fcf-4e76-98c9-2289473eade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:51.533965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:51.550771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 94/184] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:51.555977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005429d2-454c-44b6-88f5-30c2ee1a8697 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:51.570826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 91/185] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:51.571567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f1af6-6fcf-4e76-98c9-2289473eade1 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:51.640948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:51.704170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005429d2-454c-44b6-88f5-30c2ee1a8697 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:51.718312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f1af6-6fcf-4e76-98c9-2289473eade1 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:51.746877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133e16d-9f45-4815-bbad-bfd3a5163a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:51.747514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133e16d-9f45-4815-bbad-bfd3a5163a0f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:51.747788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133e16d-9f45-4815-bbad-bfd3a5163a0f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:51.748057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133e16d-9f45-4815-bbad-bfd3a5163a0f None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:51.749213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133e16d-9f45-4815-bbad-bfd3a5163a0f None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:51.751338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133e16d-9f45-4815-bbad-bfd3a5163a0f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:51.754535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 95/186] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:51 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:51.754535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133e16d-9f45-4815-bbad-bfd3a5163a0f None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:51.765977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ff29e-35ac-4784-9614-508295b97214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:51.766671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ff29e-35ac-4784-9614-508295b97214 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:51.766953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ff29e-35ac-4784-9614-508295b97214 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:51.767227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ff29e-35ac-4784-9614-508295b97214 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:51.782235 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ff29e-35ac-4784-9614-508295b97214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-151ff29e-35ac-4784-9614-508295b97214 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:51.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:51.791061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 92/187] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:51.792000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ff29e-35ac-4784-9614-508295b97214 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:51.852801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:51.882852 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133e16d-9f45-4815-bbad-bfd3a5163a0f None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:51.910154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:51.919066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ff29e-35ac-4784-9614-508295b97214 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:51.922745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062ec34-f81c-4649-8732-9ee24e3691eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:51.923621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062ec34-f81c-4649-8732-9ee24e3691eb None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:51.923899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062ec34-f81c-4649-8732-9ee24e3691eb None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:51.924170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062ec34-f81c-4649-8732-9ee24e3691eb None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:51.924855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062ec34-f81c-4649-8732-9ee24e3691eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:51.925156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062ec34-f81c-4649-8732-9ee24e3691eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:51.926594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062ec34-f81c-4649-8732-9ee24e3691eb None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:51.936309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 96/188] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:51.939525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062ec34-f81c-4649-8732-9ee24e3691eb None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:51.956419 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100a39b-8101-4250-abc9-d6d8669fe1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:51.957062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a39b-8101-4250-abc9-d6d8669fe1ad None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:51.957343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a39b-8101-4250-abc9-d6d8669fe1ad None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:51.957621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a39b-8101-4250-abc9-d6d8669fe1ad None admin] PATH_INFO: `/v3/projects/3cd1942bafc649eaa8bbd93187a3a618` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:51.959145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100a39b-8101-4250-abc9-d6d8669fe1ad None admin] RBAC: Authorizing `identity:get_project(project_id=3cd1942bafc649eaa8bbd93187a3a618)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:51.960644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100a39b-8101-4250-abc9-d6d8669fe1ad None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:51.962466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 93/189] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:51 2022] GET /identity/v3/projects/3cd1942bafc649eaa8bbd93187a3a618 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:51.963359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100a39b-8101-4250-abc9-d6d8669fe1ad None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:52.058531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062ec34-f81c-4649-8732-9ee24e3691eb None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:52.085909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100a39b-8101-4250-abc9-d6d8669fe1ad None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:52.099865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0921ff-ab1b-4e56-a283-235c2ee05bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:52.100645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:52.100821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0921ff-ab1b-4e56-a283-235c2ee05bd9 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:52.100954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0921ff-ab1b-4e56-a283-235c2ee05bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:52.101193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0921ff-ab1b-4e56-a283-235c2ee05bd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:52.113496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0921ff-ab1b-4e56-a283-235c2ee05bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-4c0921ff-ab1b-4e56-a283-235c2ee05bd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:52.119595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:52.120916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 97/190] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:04:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:52.122445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0921ff-ab1b-4e56-a283-235c2ee05bd9 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:52.125464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fc56d-6075-42bb-893c-7a8c6344303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:52.126315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fc56d-6075-42bb-893c-7a8c6344303f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:52.126630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fc56d-6075-42bb-893c-7a8c6344303f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:52.126898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fc56d-6075-42bb-893c-7a8c6344303f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:52.127827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fc56d-6075-42bb-893c-7a8c6344303f None admin] RBAC: Adding query filter params (role.id=c2fe89b1b9fa47b19e02efd64ace9000, scope.project.id=c84516b7267f4b43bf816381e3e60ae6, user.id=734f29275e5547fcb788acba34124924) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:52.128134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fc56d-6075-42bb-893c-7a8c6344303f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:52.129591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fc56d-6075-42bb-893c-7a8c6344303f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:52.147288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 94/191] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:51 2022] GET /identity/v3/role_assignments?user.id=734f29275e5547fcb788acba34124924&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=c2fe89b1b9fa47b19e02efd64ace9000 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:52.147288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fc56d-6075-42bb-893c-7a8c6344303f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:52.185231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:52.256564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0921ff-ab1b-4e56-a283-235c2ee05bd9 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:52.272391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fc56d-6075-42bb-893c-7a8c6344303f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:52.296693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec4875-b3ce-4f49-a34f-abb592372be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:52.297325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec4875-b3ce-4f49-a34f-abb592372be5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:52.297602 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec4875-b3ce-4f49-a34f-abb592372be5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:52.298738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec4875-b3ce-4f49-a34f-abb592372be5 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:52.298865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec4875-b3ce-4f49-a34f-abb592372be5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:52.299115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec4875-b3ce-4f49-a34f-abb592372be5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:52.300528 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec4875-b3ce-4f49-a34f-abb592372be5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:52.312060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 98/192] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:52.312916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec4875-b3ce-4f49-a34f-abb592372be5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:52.318680 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f221d-87d1-44bd-924b-9bc51b87471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:52.319593 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f221d-87d1-44bd-924b-9bc51b87471c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:52.320062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f221d-87d1-44bd-924b-9bc51b87471c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:52.320334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f221d-87d1-44bd-924b-9bc51b87471c None admin] PATH_INFO: `/v3/domains/default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:52.321498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f221d-87d1-44bd-924b-9bc51b87471c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:52.322997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f221d-87d1-44bd-924b-9bc51b87471c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:52.324615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 95/193] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:52.325389 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f221d-87d1-44bd-924b-9bc51b87471c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:52.376922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:52.436495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f221d-87d1-44bd-924b-9bc51b87471c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:52.443024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec4875-b3ce-4f49-a34f-abb592372be5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:52.452259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:52.471709 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dd860-9035-492b-8371-ec1147d6c62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:52.472318 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dd860-9035-492b-8371-ec1147d6c62b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:52.472597 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dd860-9035-492b-8371-ec1147d6c62b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:52.472871 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dd860-9035-492b-8371-ec1147d6c62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:52.474123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dd860-9035-492b-8371-ec1147d6c62b None admin] RBAC: Adding query filter params (role.id=13accf89c12040508974db47c6905b31, scope.project.id=3cd1942bafc649eaa8bbd93187a3a618, user.id=734f29275e5547fcb788acba34124924) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:52.474435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dd860-9035-492b-8371-ec1147d6c62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:52.475923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dd860-9035-492b-8371-ec1147d6c62b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:52.481812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 96/194] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:52 2022] GET /identity/v3/role_assignments?user.id=734f29275e5547fcb788acba34124924&scope.project.id=3cd1942bafc649eaa8bbd93187a3a618&role.id=13accf89c12040508974db47c6905b31 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:52.482647 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dd860-9035-492b-8371-ec1147d6c62b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:52.486899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb44ae5-7022-43f7-92d4-a9e388c5f6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:52.487567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44ae5-7022-43f7-92d4-a9e388c5f6c2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:52.487873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44ae5-7022-43f7-92d4-a9e388c5f6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:52.488148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44ae5-7022-43f7-92d4-a9e388c5f6c2 None admin] PATH_INFO: `/v3/users/b678139a68144bebaab2c468d7348a15` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:52.513779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb44ae5-7022-43f7-92d4-a9e388c5f6c2 None admin] RBAC: Authorizing `identity:get_user(user_id=b678139a68144bebaab2c468d7348a15)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:52.515338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb44ae5-7022-43f7-92d4-a9e388c5f6c2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:52.517131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 99/195] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:52 2022] GET /identity/v3/users/b678139a68144bebaab2c468d7348a15 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:52.523513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb44ae5-7022-43f7-92d4-a9e388c5f6c2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:52.627286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dd860-9035-492b-8371-ec1147d6c62b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:52.639751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:52.653672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb44ae5-7022-43f7-92d4-a9e388c5f6c2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:52.671088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed0021-fc5a-4ad8-83aa-97ec51a1b4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:52.671882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed0021-fc5a-4ad8-83aa-97ec51a1b4a7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:52.672692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed0021-fc5a-4ad8-83aa-97ec51a1b4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:52.672865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed0021-fc5a-4ad8-83aa-97ec51a1b4a7 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:52.674920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed0021-fc5a-4ad8-83aa-97ec51a1b4a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:52.675329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed0021-fc5a-4ad8-83aa-97ec51a1b4a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:52.677896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed0021-fc5a-4ad8-83aa-97ec51a1b4a7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:52.692800 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 97/196] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:04:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:52.694849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed0021-fc5a-4ad8-83aa-97ec51a1b4a7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:52.699481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93500d3b-9ada-4cdb-bef8-67cce365b4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:52.700467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93500d3b-9ada-4cdb-bef8-67cce365b4ff None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:52.700935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93500d3b-9ada-4cdb-bef8-67cce365b4ff None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:52.701213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93500d3b-9ada-4cdb-bef8-67cce365b4ff None admin] PATH_INFO: `/v3/users/f766fe03e07644c5b1f918e54a832d4d` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:52.703604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93500d3b-9ada-4cdb-bef8-67cce365b4ff None admin] RBAC: Authorizing `identity:get_user(user_id=f766fe03e07644c5b1f918e54a832d4d)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:52.706146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93500d3b-9ada-4cdb-bef8-67cce365b4ff None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:52.709910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 100/197] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:52 2022] GET /identity/v3/users/f766fe03e07644c5b1f918e54a832d4d => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:52.711305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93500d3b-9ada-4cdb-bef8-67cce365b4ff None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:52.751425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:52.820013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93500d3b-9ada-4cdb-bef8-67cce365b4ff None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:52.821146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed0021-fc5a-4ad8-83aa-97ec51a1b4a7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:52.855954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7a821-2dec-432f-8b39-8da128771969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:52.856173 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe76417-4b6a-4303-ba90-e0f804856d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:52.856561 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7a821-2dec-432f-8b39-8da128771969 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:52.856678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe76417-4b6a-4303-ba90-e0f804856d23 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:52.856838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7a821-2dec-432f-8b39-8da128771969 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:52.856924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe76417-4b6a-4303-ba90-e0f804856d23 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:52.857106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7a821-2dec-432f-8b39-8da128771969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:52.857190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe76417-4b6a-4303-ba90-e0f804856d23 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:52.857843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe76417-4b6a-4303-ba90-e0f804856d23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:52.859388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe76417-4b6a-4303-ba90-e0f804856d23 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:52.926255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7a821-2dec-432f-8b39-8da128771969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:52.928564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7a821-2dec-432f-8b39-8da128771969 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:52.961240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:52.982446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 98/198] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:04:52 2022] POST /identity/v3/users => generated 297 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:04:52.987568 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe76417-4b6a-4303-ba90-e0f804856d23 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:53.034803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:53.044398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 101/199] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:53.044398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7a821-2dec-432f-8b39-8da128771969 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:53.127260 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe76417-4b6a-4303-ba90-e0f804856d23 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:53.169247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00bd79-8977-4235-a72d-55a3089c68da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:53.169843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00bd79-8977-4235-a72d-55a3089c68da None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:53.170119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00bd79-8977-4235-a72d-55a3089c68da None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:53.170814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00bd79-8977-4235-a72d-55a3089c68da None admin] PATH_INFO: `/v3/users/b678139a68144bebaab2c468d7348a15` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:53.172336 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00bd79-8977-4235-a72d-55a3089c68da None admin] RBAC: Authorizing `identity:get_user(user_id=b678139a68144bebaab2c468d7348a15)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:53.173671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00bd79-8977-4235-a72d-55a3089c68da None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:53.175705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 99/200] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:52 2022] GET /identity/v3/users/b678139a68144bebaab2c468d7348a15 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:53.175705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00bd79-8977-4235-a72d-55a3089c68da None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:53.189484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7a821-2dec-432f-8b39-8da128771969 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:53.240511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268e30d-0d47-4b6f-bbd6-3234b5b97e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:53.241352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268e30d-0d47-4b6f-bbd6-3234b5b97e42 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:53.241460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268e30d-0d47-4b6f-bbd6-3234b5b97e42 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:53.241660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268e30d-0d47-4b6f-bbd6-3234b5b97e42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:53.245218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:53.301350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:53.309600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00bd79-8977-4235-a72d-55a3089c68da None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:53.332932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268e30d-0d47-4b6f-bbd6-3234b5b97e42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:53.335495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268e30d-0d47-4b6f-bbd6-3234b5b97e42 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:53.363377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea96bc-3ef5-4f7d-9938-5f8d6f4003d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:53.364154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea96bc-3ef5-4f7d-9938-5f8d6f4003d7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:53.364591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea96bc-3ef5-4f7d-9938-5f8d6f4003d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:53.364912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea96bc-3ef5-4f7d-9938-5f8d6f4003d7 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:53.366088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea96bc-3ef5-4f7d-9938-5f8d6f4003d7 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:53.367570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea96bc-3ef5-4f7d-9938-5f8d6f4003d7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:53.369238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 100/201] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:53 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:53.372374 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea96bc-3ef5-4f7d-9938-5f8d6f4003d7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:53.487990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 102/202] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:53.494699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268e30d-0d47-4b6f-bbd6-3234b5b97e42 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:53.509118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea96bc-3ef5-4f7d-9938-5f8d6f4003d7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:53.543158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:53.549724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00defe3-8830-4ff2-84bb-d869b540eecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:53.557384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00defe3-8830-4ff2-84bb-d869b540eecc None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:53.557384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00defe3-8830-4ff2-84bb-d869b540eecc None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:53.557384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00defe3-8830-4ff2-84bb-d869b540eecc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:53.587438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:53.641453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00defe3-8830-4ff2-84bb-d869b540eecc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:53.641877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268e30d-0d47-4b6f-bbd6-3234b5b97e42 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:53.643060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00defe3-8830-4ff2-84bb-d869b540eecc None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:53.690450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51307280-063c-48e6-aef4-3d1083d3a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:53.692336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51307280-063c-48e6-aef4-3d1083d3a953 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:53.693122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51307280-063c-48e6-aef4-3d1083d3a953 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:53.693617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51307280-063c-48e6-aef4-3d1083d3a953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:53.694887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51307280-063c-48e6-aef4-3d1083d3a953 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=0a33f37b0d0d4ced9878417f46335942, user.id=b678139a68144bebaab2c468d7348a15) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:53.695545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51307280-063c-48e6-aef4-3d1083d3a953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:53.697986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51307280-063c-48e6-aef4-3d1083d3a953 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:53.710747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 103/203] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:53 2022] GET /identity/v3/role_assignments?user.id=b678139a68144bebaab2c468d7348a15&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=4b10782d99b74f5bb453b82286012e0c => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:53.710747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51307280-063c-48e6-aef4-3d1083d3a953 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:53.853980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 101/204] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:53.859651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00defe3-8830-4ff2-84bb-d869b540eecc None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:53.861880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:53.872103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51307280-063c-48e6-aef4-3d1083d3a953 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:53.931444 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1400a-68a4-4083-9270-cb3202e143bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:53.932484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1400a-68a4-4083-9270-cb3202e143bc None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:53.932957 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1400a-68a4-4083-9270-cb3202e143bc None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:53.933418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1400a-68a4-4083-9270-cb3202e143bc None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:53.935812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1400a-68a4-4083-9270-cb3202e143bc None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:53.937499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1400a-68a4-4083-9270-cb3202e143bc None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:53.939621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 104/205] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:53 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:53.947806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1400a-68a4-4083-9270-cb3202e143bc None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:53.972706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:54.041154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00defe3-8830-4ff2-84bb-d869b540eecc None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:54.112071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f2ea5-279c-4a95-87d7-5f57e48f6def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:54.112644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f2ea5-279c-4a95-87d7-5f57e48f6def None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:54.112917 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f2ea5-279c-4a95-87d7-5f57e48f6def None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:54.113183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f2ea5-279c-4a95-87d7-5f57e48f6def None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:54.115563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f2ea5-279c-4a95-87d7-5f57e48f6def None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:54.117243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f2ea5-279c-4a95-87d7-5f57e48f6def None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:54.119380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 102/206] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:53 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:54.126857 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f2ea5-279c-4a95-87d7-5f57e48f6def None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:54.152364 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1400a-68a4-4083-9270-cb3202e143bc None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:54.221929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805e225-f198-4c40-9b1b-e312f8a6b632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:54.226526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805e225-f198-4c40-9b1b-e312f8a6b632 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:54.226526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805e225-f198-4c40-9b1b-e312f8a6b632 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:54.226526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805e225-f198-4c40-9b1b-e312f8a6b632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:54.226526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805e225-f198-4c40-9b1b-e312f8a6b632 None admin] RBAC: Adding query filter params (role.id=ef17b1e922a942a083c0f1905c8c523c, scope.project.id=c84516b7267f4b43bf816381e3e60ae6, user.id=f766fe03e07644c5b1f918e54a832d4d) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:54.226526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805e225-f198-4c40-9b1b-e312f8a6b632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:54.226526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805e225-f198-4c40-9b1b-e312f8a6b632 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:54.241124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 105/207] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:53 2022] GET /identity/v3/role_assignments?user.id=f766fe03e07644c5b1f918e54a832d4d&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=ef17b1e922a942a083c0f1905c8c523c => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:54.268947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f2ea5-279c-4a95-87d7-5f57e48f6def None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:54.312094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:54.324776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d5c80-029b-4974-ac03-599d1fcb210f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:54.329401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d5c80-029b-4974-ac03-599d1fcb210f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:54.329401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d5c80-029b-4974-ac03-599d1fcb210f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:54.329401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d5c80-029b-4974-ac03-599d1fcb210f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:54.329401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d5c80-029b-4974-ac03-599d1fcb210f None admin] RBAC: Adding query filter params (role.id=c2fe89b1b9fa47b19e02efd64ace9000, scope.project.id=0a33f37b0d0d4ced9878417f46335942, user.id=b678139a68144bebaab2c468d7348a15) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:54.329401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d5c80-029b-4974-ac03-599d1fcb210f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:54.329401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d5c80-029b-4974-ac03-599d1fcb210f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:54.340114 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 103/208] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:54 2022] GET /identity/v3/role_assignments?user.id=b678139a68144bebaab2c468d7348a15&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=c2fe89b1b9fa47b19e02efd64ace9000 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:55.250413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68a84f-0986-4301-945e-6b834247605d None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:55.250994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68a84f-0986-4301-945e-6b834247605d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:55.251084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68a84f-0986-4301-945e-6b834247605d None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:55.252691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 106/209] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:55.258526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f57e-c3b4-4b01-91d0-bf2f31bd3c65 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:55.258976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f57e-c3b4-4b01-91d0-bf2f31bd3c65 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:55.259158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f57e-c3b4-4b01-91d0-bf2f31bd3c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:55.278084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-67b3f57e-c3b4-4b01-91d0-bf2f31bd3c65 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:55.279327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-67b3f57e-c3b4-4b01-91d0-bf2f31bd3c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:55.351853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb33973-3004-45f7-a4fa-5c9728b41714 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:55.352174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb33973-3004-45f7-a4fa-5c9728b41714 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:55.352399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb33973-3004-45f7-a4fa-5c9728b41714 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:55.353742 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 107/210] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:04:55.357484 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 104/211] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:55.362134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f711e-9ffe-4b03-a354-27b48894b9a0 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:55.362134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f711e-9ffe-4b03-a354-27b48894b9a0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:55.362134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f711e-9ffe-4b03-a354-27b48894b9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:55.380959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-788f711e-9ffe-4b03-a354-27b48894b9a0 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:55.382234 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-788f711e-9ffe-4b03-a354-27b48894b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:55.409135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:55.457420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 108/212] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:04:55.652818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3f57e-c3b4-4b01-91d0-bf2f31bd3c65 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:55.737818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f711e-9ffe-4b03-a354-27b48894b9a0 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:55.761612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3f57e-c3b4-4b01-91d0-bf2f31bd3c65 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:55.799886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9b340-2462-4699-9c67-a1c140260305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:55.800759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9b340-2462-4699-9c67-a1c140260305 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:55.801252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9b340-2462-4699-9c67-a1c140260305 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:55.802952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9b340-2462-4699-9c67-a1c140260305 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:55.804252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9b340-2462-4699-9c67-a1c140260305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:55.805843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9b340-2462-4699-9c67-a1c140260305 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:55.807632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 105/213] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:55.808275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:55.815332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9b340-2462-4699-9c67-a1c140260305 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:55.871728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f711e-9ffe-4b03-a354-27b48894b9a0 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:55.912951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f19994-368a-49df-bb03-3a2df5e3c197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:55.913623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f19994-368a-49df-bb03-3a2df5e3c197 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:55.913899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f19994-368a-49df-bb03-3a2df5e3c197 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:55.914169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f19994-368a-49df-bb03-3a2df5e3c197 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:55.927569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f19994-368a-49df-bb03-3a2df5e3c197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-c1f19994-368a-49df-bb03-3a2df5e3c197 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:55.932946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:55.933845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:55.934682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 109/214] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:04:55.935827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d0651-66c9-4500-8439-7d10acf1c07b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:55.935827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d0651-66c9-4500-8439-7d10acf1c07b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:55.935986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d0651-66c9-4500-8439-7d10acf1c07b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:55.937410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 110/215] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:55.940815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d0651-66c9-4500-8439-7d10acf1c07b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:55.950339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9b340-2462-4699-9c67-a1c140260305 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:55.969448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:55.988211 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be81237-3972-4289-847a-4b422a99b32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:55.989120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be81237-3972-4289-847a-4b422a99b32e None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:55.989373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be81237-3972-4289-847a-4b422a99b32e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:55.989474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be81237-3972-4289-847a-4b422a99b32e None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:55.990431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be81237-3972-4289-847a-4b422a99b32e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:55.993224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be81237-3972-4289-847a-4b422a99b32e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:56.054702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d0651-66c9-4500-8439-7d10acf1c07b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:56.089560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb06dc1-cb91-471f-8870-e1042cdaf35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:56.090165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb06dc1-cb91-471f-8870-e1042cdaf35f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:56.090442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb06dc1-cb91-471f-8870-e1042cdaf35f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:56.090738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb06dc1-cb91-471f-8870-e1042cdaf35f None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:56.091399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb06dc1-cb91-471f-8870-e1042cdaf35f None admin] RBAC: Adding query filter params (name=member) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:56.091696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb06dc1-cb91-471f-8870-e1042cdaf35f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:56.093052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb06dc1-cb91-471f-8870-e1042cdaf35f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:56.101958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 111/216] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:56.106609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce996-0339-4037-ba99-c9499f0e82ee None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:56.106609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce996-0339-4037-ba99-c9499f0e82ee None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:56.106609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce996-0339-4037-ba99-c9499f0e82ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:56.124048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 106/217] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:04:55 2022] POST /identity/v3/users => generated 297 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:04:56.124363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-7f6ce996-0339-4037-ba99-c9499f0e82ee None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:56.125591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7f6ce996-0339-4037-ba99-c9499f0e82ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:56.126347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be81237-3972-4289-847a-4b422a99b32e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:56.197381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:56.215053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 112/218] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:56.257344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be81237-3972-4289-847a-4b422a99b32e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:56.296622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed821f40-e888-4019-88a3-338f76a6f359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:56.297872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed821f40-e888-4019-88a3-338f76a6f359 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:56.298412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed821f40-e888-4019-88a3-338f76a6f359 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:56.298711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed821f40-e888-4019-88a3-338f76a6f359 None admin] PATH_INFO: `/v3/users/f8ac338975bf4f49ab03504322ee14bf` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:56.314713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed821f40-e888-4019-88a3-338f76a6f359 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ac338975bf4f49ab03504322ee14bf)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:56.316473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed821f40-e888-4019-88a3-338f76a6f359 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:56.319097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 107/219] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:56 2022] GET /identity/v3/users/f8ac338975bf4f49ab03504322ee14bf => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:56.324972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ce996-0339-4037-ba99-c9499f0e82ee None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:56.348264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:56.391971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9aa85-f04c-436d-a8fd-705e6947fa37 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:56.392287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9aa85-f04c-436d-a8fd-705e6947fa37 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:56.392519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9aa85-f04c-436d-a8fd-705e6947fa37 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:56.393898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 108/220] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:56.399944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221c158-2f95-4dc9-9d31-8e57b8bda67b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:56.400188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221c158-2f95-4dc9-9d31-8e57b8bda67b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:56.400464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221c158-2f95-4dc9-9d31-8e57b8bda67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:56.422069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-e221c158-2f95-4dc9-9d31-8e57b8bda67b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:56.423869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e221c158-2f95-4dc9-9d31-8e57b8bda67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:56.509165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ce996-0339-4037-ba99-c9499f0e82ee None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:56.524960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 109/221] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:04:56.526780 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221c158-2f95-4dc9-9d31-8e57b8bda67b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:56.554084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e05d5c-d181-4c60-8da8-d9bb5b40dcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:56.554744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e05d5c-d181-4c60-8da8-d9bb5b40dcbf None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:56.555049 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e05d5c-d181-4c60-8da8-d9bb5b40dcbf None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:56.555295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e05d5c-d181-4c60-8da8-d9bb5b40dcbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:56.576061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:56.653805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e05d5c-d181-4c60-8da8-d9bb5b40dcbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:56.655322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e05d5c-d181-4c60-8da8-d9bb5b40dcbf None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:56.670788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:56.683231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221c158-2f95-4dc9-9d31-8e57b8bda67b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:56.740074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5606b96-981b-4798-9240-564a2316112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:56.741068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5606b96-981b-4798-9240-564a2316112c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:56.741635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5606b96-981b-4798-9240-564a2316112c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:56.742122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5606b96-981b-4798-9240-564a2316112c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:56.762052 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5606b96-981b-4798-9240-564a2316112c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:56.769675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 113/222] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-e5606b96-981b-4798-9240-564a2316112c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:56.772930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:56.777117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:04:56.778215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:56.779000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e05d5c-d181-4c60-8da8-d9bb5b40dcbf None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:56.782579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 110/223] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:04:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:56.790460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5606b96-981b-4798-9240-564a2316112c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:56.892350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e05d5c-d181-4c60-8da8-d9bb5b40dcbf None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:56.907507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:56.935479 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d46c0-00f0-4c83-ad6a-d14f8eb66d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:56.936325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d46c0-00f0-4c83-ad6a-d14f8eb66d7b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:56.936701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d46c0-00f0-4c83-ad6a-d14f8eb66d7b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:56.937070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d46c0-00f0-4c83-ad6a-d14f8eb66d7b None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:56.938366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d46c0-00f0-4c83-ad6a-d14f8eb66d7b None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:56.939951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d46c0-00f0-4c83-ad6a-d14f8eb66d7b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:56.943235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 114/224] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:56 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:56.944200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d46c0-00f0-4c83-ad6a-d14f8eb66d7b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:56.974883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:56.980126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5606b96-981b-4798-9240-564a2316112c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:57.017789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bad27e-faee-40ba-9f36-67517ddd0fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:57.018716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad27e-faee-40ba-9f36-67517ddd0fb3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:57.019179 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad27e-faee-40ba-9f36-67517ddd0fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:57.019649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad27e-faee-40ba-9f36-67517ddd0fb3 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:57.020558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bad27e-faee-40ba-9f36-67517ddd0fb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:57.021041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bad27e-faee-40ba-9f36-67517ddd0fb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:57.022728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bad27e-faee-40ba-9f36-67517ddd0fb3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:57.032121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 111/225] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:57.033399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bad27e-faee-40ba-9f36-67517ddd0fb3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:57.080078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d46c0-00f0-4c83-ad6a-d14f8eb66d7b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:57.118331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb664f9-e132-4091-913c-90a0223ad82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:57.119036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb664f9-e132-4091-913c-90a0223ad82d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:57.119313 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb664f9-e132-4091-913c-90a0223ad82d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:57.119582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb664f9-e132-4091-913c-90a0223ad82d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:57.131758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb664f9-e132-4091-913c-90a0223ad82d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-3bb664f9-e132-4091-913c-90a0223ad82d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:57.137035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:57.137960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 115/226] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:04:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:57.143529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb664f9-e132-4091-913c-90a0223ad82d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:57.156280 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bad27e-faee-40ba-9f36-67517ddd0fb3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:57.186801 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:57.194037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77e2e3-6166-450b-913f-981877a0208a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:57.194895 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77e2e3-6166-450b-913f-981877a0208a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:57.195316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77e2e3-6166-450b-913f-981877a0208a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:57.201162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77e2e3-6166-450b-913f-981877a0208a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:57.201162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77e2e3-6166-450b-913f-981877a0208a None admin] RBAC: Adding query filter params (role.id=13accf89c12040508974db47c6905b31, scope.project.id=0a33f37b0d0d4ced9878417f46335942, user.id=f8ac338975bf4f49ab03504322ee14bf) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:57.201162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77e2e3-6166-450b-913f-981877a0208a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:57.201162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77e2e3-6166-450b-913f-981877a0208a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:57.204983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 112/227] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:04:57 2022] GET /identity/v3/role_assignments?user.id=f8ac338975bf4f49ab03504322ee14bf&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=13accf89c12040508974db47c6905b31 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:57.206248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77e2e3-6166-450b-913f-981877a0208a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:57.234375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:57.256861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb664f9-e132-4091-913c-90a0223ad82d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:57.303406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37ec83-3f5e-4b63-86c7-36b3ca4f97f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:57.304805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ec83-3f5e-4b63-86c7-36b3ca4f97f5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:57.305093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ec83-3f5e-4b63-86c7-36b3ca4f97f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:57.305368 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ec83-3f5e-4b63-86c7-36b3ca4f97f5 None admin] PATH_INFO: `/v3/users/b678139a68144bebaab2c468d7348a15` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:57.308930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37ec83-3f5e-4b63-86c7-36b3ca4f97f5 None admin] RBAC: Authorizing `identity:get_user(user_id=b678139a68144bebaab2c468d7348a15)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:57.310423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37ec83-3f5e-4b63-86c7-36b3ca4f97f5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:57.317508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 116/228] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:57 2022] GET /identity/v3/users/b678139a68144bebaab2c468d7348a15 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:57.319566 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37ec83-3f5e-4b63-86c7-36b3ca4f97f5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:57.400188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77e2e3-6166-450b-913f-981877a0208a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:57.449587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37ec83-3f5e-4b63-86c7-36b3ca4f97f5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:57.456997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcd00d-5cfb-4cce-9dc8-5e0d1451a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:57.458602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcd00d-5cfb-4cce-9dc8-5e0d1451a359 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:57.459229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcd00d-5cfb-4cce-9dc8-5e0d1451a359 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:57.459706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcd00d-5cfb-4cce-9dc8-5e0d1451a359 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:57.460591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcd00d-5cfb-4cce-9dc8-5e0d1451a359 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:57.461093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcd00d-5cfb-4cce-9dc8-5e0d1451a359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:57.461310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:57.463341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcd00d-5cfb-4cce-9dc8-5e0d1451a359 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:57.473050 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 113/229] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:04:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:57.480151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcd00d-5cfb-4cce-9dc8-5e0d1451a359 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:57.493252 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ab408-0f70-4f25-98b9-737d6c1a4640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:57.493968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ab408-0f70-4f25-98b9-737d6c1a4640 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:57.494309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ab408-0f70-4f25-98b9-737d6c1a4640 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:57.494618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ab408-0f70-4f25-98b9-737d6c1a4640 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:57.496034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ab408-0f70-4f25-98b9-737d6c1a4640 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:57.499049 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ab408-0f70-4f25-98b9-737d6c1a4640 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:57.500972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 117/230] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:57 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:57.511625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ab408-0f70-4f25-98b9-737d6c1a4640 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:57.611292 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcd00d-5cfb-4cce-9dc8-5e0d1451a359 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:57.642272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:57.652031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73a6af-b916-4015-bd77-86ae1e5bfc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:57.652677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73a6af-b916-4015-bd77-86ae1e5bfc39 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:57.652956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73a6af-b916-4015-bd77-86ae1e5bfc39 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:57.653227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73a6af-b916-4015-bd77-86ae1e5bfc39 None admin] PATH_INFO: `/v3/users/b678139a68144bebaab2c468d7348a15` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:57.654411 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73a6af-b916-4015-bd77-86ae1e5bfc39 None admin] RBAC: Authorizing `identity:get_user(user_id=b678139a68144bebaab2c468d7348a15)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:57.655909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73a6af-b916-4015-bd77-86ae1e5bfc39 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:57.657653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 114/231] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:57 2022] GET /identity/v3/users/b678139a68144bebaab2c468d7348a15 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:57.663835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73a6af-b916-4015-bd77-86ae1e5bfc39 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:57.782867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73a6af-b916-4015-bd77-86ae1e5bfc39 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:57.807350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ab408-0f70-4f25-98b9-737d6c1a4640 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:57.821633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37796ed4-d3d0-4c75-bdd3-265a43590baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:57.822283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37796ed4-d3d0-4c75-bdd3-265a43590baa None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:57.822611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37796ed4-d3d0-4c75-bdd3-265a43590baa None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:57.822898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37796ed4-d3d0-4c75-bdd3-265a43590baa None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:57.824121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37796ed4-d3d0-4c75-bdd3-265a43590baa None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:57.825589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37796ed4-d3d0-4c75-bdd3-265a43590baa None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:57.830606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 115/232] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:04:57 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:57.838502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37796ed4-d3d0-4c75-bdd3-265a43590baa None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:57.859198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:57.892894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32620615-250d-4afd-8bc3-c0513026678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:57.893915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32620615-250d-4afd-8bc3-c0513026678d None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:57.894393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32620615-250d-4afd-8bc3-c0513026678d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:57.894904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32620615-250d-4afd-8bc3-c0513026678d None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942/users/b678139a68144bebaab2c468d7348a15/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:57.904073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32620615-250d-4afd-8bc3-c0513026678d None admin] RBAC: Authorizing `identity:create_grant(project_id=0a33f37b0d0d4ced9878417f46335942, user_id=b678139a68144bebaab2c468d7348a15, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:57.906057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32620615-250d-4afd-8bc3-c0513026678d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:57.949115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37796ed4-d3d0-4c75-bdd3-265a43590baa None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:57.986898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dee02-aa49-413b-9003-02e35a007a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:57.987567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dee02-aa49-413b-9003-02e35a007a4c None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:57.987847 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dee02-aa49-413b-9003-02e35a007a4c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:57.988118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dee02-aa49-413b-9003-02e35a007a4c None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942/users/b678139a68144bebaab2c468d7348a15/roles/c2fe89b1b9fa47b19e02efd64ace9000` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:57.991218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72dee02-aa49-413b-9003-02e35a007a4c None admin] RBAC: Authorizing `identity:create_grant(project_id=0a33f37b0d0d4ced9878417f46335942, user_id=b678139a68144bebaab2c468d7348a15, role_id=c2fe89b1b9fa47b19e02efd64ace9000)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:57.992863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72dee02-aa49-413b-9003-02e35a007a4c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:57.993870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 118/233] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:04:57 2022] PUT /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942/users/b678139a68144bebaab2c468d7348a15/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:04:58.092082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 116/234] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:04:57 2022] PUT /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942/users/b678139a68144bebaab2c468d7348a15/roles/c2fe89b1b9fa47b19e02efd64ace9000 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:04:58.700459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b00c7-1488-4fbf-bdf2-f7c82ecda7b8 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:58.700999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b00c7-1488-4fbf-bdf2-f7c82ecda7b8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:58.700999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b00c7-1488-4fbf-bdf2-f7c82ecda7b8 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:58.702640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 119/235] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:58.707805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c521e-6f4c-43d8-a5d9-9b626682bfcb None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:58.708795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c521e-6f4c-43d8-a5d9-9b626682bfcb None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:58.709307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c521e-6f4c-43d8-a5d9-9b626682bfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:58.734047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-394c521e-6f4c-43d8-a5d9-9b626682bfcb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:58.735714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-394c521e-6f4c-43d8-a5d9-9b626682bfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:58.784951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e45ba1-5761-4815-b195-5c3e125c347b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:58.786105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e45ba1-5761-4815-b195-5c3e125c347b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:58.786620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e45ba1-5761-4815-b195-5c3e125c347b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:58.788731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 120/236] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:58.800097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3c3cb-bd92-4ee4-a8a5-5cf369e940c1 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:58.800764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3c3cb-bd92-4ee4-a8a5-5cf369e940c1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:58.801903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3c3cb-bd92-4ee4-a8a5-5cf369e940c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:58.812758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 117/237] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:58.838415 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8ca3c3cb-bd92-4ee4-a8a5-5cf369e940c1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:58.839676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8ca3c3cb-bd92-4ee4-a8a5-5cf369e940c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:58.879616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:58.920810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 121/238] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:59.148985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c521e-6f4c-43d8-a5d9-9b626682bfcb None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:59.162467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:59.225607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3c3cb-bd92-4ee4-a8a5-5cf369e940c1 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:59.284871 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c521e-6f4c-43d8-a5d9-9b626682bfcb None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:59.323560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54db9f-ce8f-4e3b-94d7-6989057855a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:59.324667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54db9f-ce8f-4e3b-94d7-6989057855a2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:59.325244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54db9f-ce8f-4e3b-94d7-6989057855a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:59.325731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54db9f-ce8f-4e3b-94d7-6989057855a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:59.337782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54db9f-ce8f-4e3b-94d7-6989057855a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-7a54db9f-ce8f-4e3b-94d7-6989057855a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:04:59.343471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:04:59.344416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:04:59.345203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:04:59.347858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 118/239] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:04:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:04:59.355427 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54db9f-ce8f-4e3b-94d7-6989057855a2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:59.381863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3c3cb-bd92-4ee4-a8a5-5cf369e940c1 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:59.422815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbaee2-0df2-4b52-a76b-b1e56253e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:59.423626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbaee2-0df2-4b52-a76b-b1e56253e72b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:59.423906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbaee2-0df2-4b52-a76b-b1e56253e72b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:59.424175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbaee2-0df2-4b52-a76b-b1e56253e72b None admin] PATH_INFO: `/v3/domains/default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:59.425292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbaee2-0df2-4b52-a76b-b1e56253e72b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:59.427544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbaee2-0df2-4b52-a76b-b1e56253e72b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:59.429451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 122/240] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:04:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:59.435002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbaee2-0df2-4b52-a76b-b1e56253e72b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:59.472521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:59.552759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbaee2-0df2-4b52-a76b-b1e56253e72b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:59.592308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12215875-2b2d-4e4d-abb1-03d093ce703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:59.592939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215875-2b2d-4e4d-abb1-03d093ce703e None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:59.593219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215875-2b2d-4e4d-abb1-03d093ce703e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:59.593496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215875-2b2d-4e4d-abb1-03d093ce703e None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:59.594158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12215875-2b2d-4e4d-abb1-03d093ce703e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:59.595940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12215875-2b2d-4e4d-abb1-03d093ce703e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:59.608403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:59.643530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54db9f-ce8f-4e3b-94d7-6989057855a2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:59.680785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 123/241] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:04:59 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:04:59.682353 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0094a-b178-43e1-83ac-970cdcf3b758 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:59.682708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0094a-b178-43e1-83ac-970cdcf3b758 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:59.682985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0094a-b178-43e1-83ac-970cdcf3b758 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:59.684354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 124/242] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:04:59.693576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed995061-62b5-4318-ad79-659cb60dae7d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:59.693880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed995061-62b5-4318-ad79-659cb60dae7d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:59.694128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed995061-62b5-4318-ad79-659cb60dae7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:59.711393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1558d7-25e6-471e-886c-3fbfdefe106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:59.712717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1558d7-25e6-471e-886c-3fbfdefe106c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:59.713186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1558d7-25e6-471e-886c-3fbfdefe106c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:59.713647 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1558d7-25e6-471e-886c-3fbfdefe106c None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:59.714555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1558d7-25e6-471e-886c-3fbfdefe106c None admin] RBAC: Adding query filter params (name=reader) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:04:59.714657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-ed995061-62b5-4318-ad79-659cb60dae7d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:04:59.715448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1558d7-25e6-471e-886c-3fbfdefe106c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:59.715783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ed995061-62b5-4318-ad79-659cb60dae7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:04:59.717264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1558d7-25e6-471e-886c-3fbfdefe106c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:59.745824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 119/243] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:04:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:04:59.751582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1558d7-25e6-471e-886c-3fbfdefe106c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:59.841897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 125/244] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:04:59.894312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1558d7-25e6-471e-886c-3fbfdefe106c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:04:59.925861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:04:59.931503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81052d9b-af10-4215-b01a-342611ebeeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:04:59.932071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81052d9b-af10-4215-b01a-342611ebeeae None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:04:59.932349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81052d9b-af10-4215-b01a-342611ebeeae None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:04:59.932622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81052d9b-af10-4215-b01a-342611ebeeae None admin] PATH_INFO: `/v3/users/44f3b7d8469e4cfaba8ecde8554d9b84` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:04:59.947676 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81052d9b-af10-4215-b01a-342611ebeeae None admin] RBAC: Authorizing `identity:get_user(user_id=44f3b7d8469e4cfaba8ecde8554d9b84)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:04:59.949155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81052d9b-af10-4215-b01a-342611ebeeae None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:04:59.950887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 120/245] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:04:59 2022] GET /identity/v3/users/44f3b7d8469e4cfaba8ecde8554d9b84 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:04:59.956716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed995061-62b5-4318-ad79-659cb60dae7d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:04:59.979928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:00.066641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed995061-62b5-4318-ad79-659cb60dae7d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:00.106186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990c7ae-6c7d-4dc4-b0e0-010a001171b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:00.106842 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990c7ae-6c7d-4dc4-b0e0-010a001171b0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:00.107118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990c7ae-6c7d-4dc4-b0e0-010a001171b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:00.107384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990c7ae-6c7d-4dc4-b0e0-010a001171b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:00.186190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990c7ae-6c7d-4dc4-b0e0-010a001171b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:00.187685 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990c7ae-6c7d-4dc4-b0e0-010a001171b0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:00.207663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81052d9b-af10-4215-b01a-342611ebeeae None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:00.215115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:00.337077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 126/246] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:04:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:00.346547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990c7ae-6c7d-4dc4-b0e0-010a001171b0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:00.363338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81052d9b-af10-4215-b01a-342611ebeeae None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:00.421518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60a804-58b3-4718-8e27-7784f00028de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:00.422182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60a804-58b3-4718-8e27-7784f00028de None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:00.422463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60a804-58b3-4718-8e27-7784f00028de None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:00.422766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60a804-58b3-4718-8e27-7784f00028de None admin] PATH_INFO: `/v3/roles/member` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:00.441099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60a804-58b3-4718-8e27-7784f00028de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-eb60a804-58b3-4718-8e27-7784f00028de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:00.452598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:00.453735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 121/247] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:00.454758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c465a50-9826-43fb-bbf0-5ad288f25b84 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:00.455502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c465a50-9826-43fb-bbf0-5ad288f25b84 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:00.455502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c465a50-9826-43fb-bbf0-5ad288f25b84 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:00.455502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 122/248] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:00.459549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c465a50-9826-43fb-bbf0-5ad288f25b84 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:00.491910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990c7ae-6c7d-4dc4-b0e0-010a001171b0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:00.521029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:00.540221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:00.541687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a3d39-0fb1-46b3-b729-f6a5cc676feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:00.542448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a3d39-0fb1-46b3-b729-f6a5cc676feb None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:00.542900 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a3d39-0fb1-46b3-b729-f6a5cc676feb None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:00.543248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a3d39-0fb1-46b3-b729-f6a5cc676feb None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:00.544752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a3d39-0fb1-46b3-b729-f6a5cc676feb None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:00.547369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a3d39-0fb1-46b3-b729-f6a5cc676feb None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:00.554517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 127/249] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:00 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:00.554517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a144d-5cf3-4429-9d13-1e2221fb8438 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:00.554517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a144d-5cf3-4429-9d13-1e2221fb8438 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:00.554517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a144d-5cf3-4429-9d13-1e2221fb8438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:00.584534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-b95a144d-5cf3-4429-9d13-1e2221fb8438 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:00.586055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b95a144d-5cf3-4429-9d13-1e2221fb8438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:00.621704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c465a50-9826-43fb-bbf0-5ad288f25b84 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:00.672172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9028f-1d6b-4801-8460-456d5028fa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:00.672804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9028f-1d6b-4801-8460-456d5028fa5a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:00.673334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9028f-1d6b-4801-8460-456d5028fa5a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:00.673599 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9028f-1d6b-4801-8460-456d5028fa5a None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:00.674279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9028f-1d6b-4801-8460-456d5028fa5a None admin] RBAC: Adding query filter params (name=member) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:00.674605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9028f-1d6b-4801-8460-456d5028fa5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:00.676515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9028f-1d6b-4801-8460-456d5028fa5a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:00.683759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 128/250] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:00.684663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a144d-5cf3-4429-9d13-1e2221fb8438 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:00.700786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 123/251] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:00.705194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9028f-1d6b-4801-8460-456d5028fa5a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:00.815197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a144d-5cf3-4429-9d13-1e2221fb8438 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:00.834854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:00.858343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ce944-b949-48c7-91ac-089ec1ae4356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:00.859198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ce944-b949-48c7-91ac-089ec1ae4356 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:00.859396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ce944-b949-48c7-91ac-089ec1ae4356 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:00.859626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ce944-b949-48c7-91ac-089ec1ae4356 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:00.860377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ce944-b949-48c7-91ac-089ec1ae4356 None admin] RBAC: Adding query filter params (role.id=ef17b1e922a942a083c0f1905c8c523c, scope.project.id=0a33f37b0d0d4ced9878417f46335942, user.id=44f3b7d8469e4cfaba8ecde8554d9b84) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:00.860683 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ce944-b949-48c7-91ac-089ec1ae4356 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:00.862146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ce944-b949-48c7-91ac-089ec1ae4356 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:00.875166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 129/252] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:00 2022] GET /identity/v3/role_assignments?user.id=44f3b7d8469e4cfaba8ecde8554d9b84&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=ef17b1e922a942a083c0f1905c8c523c => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:00.877602 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569ed2f-bd2f-45fa-a31a-2d02ab4cd88d None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:00.877886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569ed2f-bd2f-45fa-a31a-2d02ab4cd88d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:00.878156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569ed2f-bd2f-45fa-a31a-2d02ab4cd88d None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:00.879893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 130/253] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:00.891236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9028f-1d6b-4801-8460-456d5028fa5a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:00.892209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9825ced-b1bf-49a6-9328-7a51c80c974d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:00.892495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9825ced-b1bf-49a6-9328-7a51c80c974d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:00.892767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9825ced-b1bf-49a6-9328-7a51c80c974d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:00.908396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:00.923173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c9825ced-b1bf-49a6-9328-7a51c80c974d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:00.924388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c9825ced-b1bf-49a6-9328-7a51c80c974d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:00.936929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e66a1-682f-4524-a222-28688cd283d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:00.937931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e66a1-682f-4524-a222-28688cd283d3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:00.938433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e66a1-682f-4524-a222-28688cd283d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:00.939014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e66a1-682f-4524-a222-28688cd283d3 None admin] PATH_INFO: `/v3/users/f8ac338975bf4f49ab03504322ee14bf` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:00.944460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e66a1-682f-4524-a222-28688cd283d3 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ac338975bf4f49ab03504322ee14bf)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:00.947827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e66a1-682f-4524-a222-28688cd283d3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:00.953303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 124/254] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:00 2022] GET /identity/v3/users/f8ac338975bf4f49ab03504322ee14bf => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:00.975984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e66a1-682f-4524-a222-28688cd283d3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:01.058827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 131/255] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:01.104405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9825ced-b1bf-49a6-9328-7a51c80c974d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:01.182546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e66a1-682f-4524-a222-28688cd283d3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:01.236500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9825ced-b1bf-49a6-9328-7a51c80c974d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:01.255442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:01.258907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e9b52-9846-4c42-a93e-6a85daa7b639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:01.259557 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e9b52-9846-4c42-a93e-6a85daa7b639 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:01.259835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e9b52-9846-4c42-a93e-6a85daa7b639 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:01.260104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e9b52-9846-4c42-a93e-6a85daa7b639 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:01.261313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e9b52-9846-4c42-a93e-6a85daa7b639 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:01.262803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e9b52-9846-4c42-a93e-6a85daa7b639 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:01.271132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 125/256] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:00 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:01.283512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e9b52-9846-4c42-a93e-6a85daa7b639 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:01.285719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863de117-b61c-4c56-81c1-764a042522d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:01.287181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863de117-b61c-4c56-81c1-764a042522d8 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:01.287703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863de117-b61c-4c56-81c1-764a042522d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:01.288164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863de117-b61c-4c56-81c1-764a042522d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:01.302757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863de117-b61c-4c56-81c1-764a042522d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-863de117-b61c-4c56-81c1-764a042522d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:01.308848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:01.309741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:05:01.310587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:01.313558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 132/257] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:05:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:01.320503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863de117-b61c-4c56-81c1-764a042522d8 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:01.398439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:01.447675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863de117-b61c-4c56-81c1-764a042522d8 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:01.500632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf1484-7369-4f17-9070-580a18cb80bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:01.501505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf1484-7369-4f17-9070-580a18cb80bd None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:01.502146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf1484-7369-4f17-9070-580a18cb80bd None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:01.503313 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf1484-7369-4f17-9070-580a18cb80bd None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:01.504519 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf1484-7369-4f17-9070-580a18cb80bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:01.506259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf1484-7369-4f17-9070-580a18cb80bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:01.509028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf1484-7369-4f17-9070-580a18cb80bd None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:01.528489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e9b52-9846-4c42-a93e-6a85daa7b639 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:01.532279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 133/258] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:01.535192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf1484-7369-4f17-9070-580a18cb80bd None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:01.579272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:01.601495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a33dc-0b8b-4e29-8cbb-a3a52d46ad25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:01.602179 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a33dc-0b8b-4e29-8cbb-a3a52d46ad25 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:01.602460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a33dc-0b8b-4e29-8cbb-a3a52d46ad25 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:01.602761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a33dc-0b8b-4e29-8cbb-a3a52d46ad25 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942/users/f8ac338975bf4f49ab03504322ee14bf/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:01.608261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a33dc-0b8b-4e29-8cbb-a3a52d46ad25 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a33f37b0d0d4ced9878417f46335942, user_id=f8ac338975bf4f49ab03504322ee14bf, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:01.609779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a33dc-0b8b-4e29-8cbb-a3a52d46ad25 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:01.673846 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf1484-7369-4f17-9070-580a18cb80bd None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:01.703808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 126/259] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:05:01 2022] PUT /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942/users/f8ac338975bf4f49ab03504322ee14bf/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:05:01.705074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a33dc-0b8b-4e29-8cbb-a3a52d46ad25 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:01.719748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7250d-ccc0-4b3b-b865-2113c49746e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:01.720406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7250d-ccc0-4b3b-b865-2113c49746e0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:01.720739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7250d-ccc0-4b3b-b865-2113c49746e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:01.721020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7250d-ccc0-4b3b-b865-2113c49746e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:01.739740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7250d-ccc0-4b3b-b865-2113c49746e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-03d7250d-ccc0-4b3b-b865-2113c49746e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:01.745495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:01.746453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:01.747332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 134/260] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:05:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:01.755009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7250d-ccc0-4b3b-b865-2113c49746e0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:01.947926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a33dc-0b8b-4e29-8cbb-a3a52d46ad25 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:01.961872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:01.968761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:01.970813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7250d-ccc0-4b3b-b865-2113c49746e0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:02.013995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91389fe9-aae4-4003-8ec3-f5680c13de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:02.014562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91389fe9-aae4-4003-8ec3-f5680c13de27 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:02.014847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91389fe9-aae4-4003-8ec3-f5680c13de27 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:02.015121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91389fe9-aae4-4003-8ec3-f5680c13de27 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:02.015802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91389fe9-aae4-4003-8ec3-f5680c13de27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:02.016103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91389fe9-aae4-4003-8ec3-f5680c13de27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:02.017492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91389fe9-aae4-4003-8ec3-f5680c13de27 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:02.028461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 135/261] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:02.034450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91389fe9-aae4-4003-8ec3-f5680c13de27 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:02.036663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a5c2b-36b5-4836-85f2-c460af489ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:02.037321 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a5c2b-36b5-4836-85f2-c460af489ed4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:02.037601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a5c2b-36b5-4836-85f2-c460af489ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:02.037875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a5c2b-36b5-4836-85f2-c460af489ed4 None admin] PATH_INFO: `/v3/users/b678139a68144bebaab2c468d7348a15` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:02.048905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a5c2b-36b5-4836-85f2-c460af489ed4 None admin] RBAC: Authorizing `identity:get_user(user_id=b678139a68144bebaab2c468d7348a15)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:02.050420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a5c2b-36b5-4836-85f2-c460af489ed4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:02.054888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 127/262] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:01 2022] GET /identity/v3/users/b678139a68144bebaab2c468d7348a15 => generated 283 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:02.058804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a5c2b-36b5-4836-85f2-c460af489ed4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:02.206200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91389fe9-aae4-4003-8ec3-f5680c13de27 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:02.259758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a5c2b-36b5-4836-85f2-c460af489ed4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:02.272107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80619840-95d5-4c65-8c9f-c4ce42e076db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:02.272673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80619840-95d5-4c65-8c9f-c4ce42e076db None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:02.272947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80619840-95d5-4c65-8c9f-c4ce42e076db None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:02.273216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80619840-95d5-4c65-8c9f-c4ce42e076db None admin] PATH_INFO: `/v3/users/b678139a68144bebaab2c468d7348a15` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:02.274513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80619840-95d5-4c65-8c9f-c4ce42e076db None admin] RBAC: Authorizing `identity:get_user(user_id=b678139a68144bebaab2c468d7348a15)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:02.275969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80619840-95d5-4c65-8c9f-c4ce42e076db None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:02.281016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 136/263] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:02 2022] GET /identity/v3/users/b678139a68144bebaab2c468d7348a15 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:02.287491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80619840-95d5-4c65-8c9f-c4ce42e076db None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:02.349620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15650303-ccd7-42dd-a445-21e65cbc2023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:02.350240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15650303-ccd7-42dd-a445-21e65cbc2023 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:02.350532 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15650303-ccd7-42dd-a445-21e65cbc2023 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:02.350803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15650303-ccd7-42dd-a445-21e65cbc2023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:02.352451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:02.454237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:02.502509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15650303-ccd7-42dd-a445-21e65cbc2023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:02.502509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15650303-ccd7-42dd-a445-21e65cbc2023 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:02.517060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80619840-95d5-4c65-8c9f-c4ce42e076db None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:02.586131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770cce7-eb59-42b6-8de8-fd9e7da28f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:02.586731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770cce7-eb59-42b6-8de8-fd9e7da28f65 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:02.587300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770cce7-eb59-42b6-8de8-fd9e7da28f65 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:02.587582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770cce7-eb59-42b6-8de8-fd9e7da28f65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:02.717289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 128/264] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:02.720376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55886b1-96ea-43de-b83c-df448028f65b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:02.720658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55886b1-96ea-43de-b83c-df448028f65b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:02.720939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55886b1-96ea-43de-b83c-df448028f65b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:02.724181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 129/265] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:02.725064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55886b1-96ea-43de-b83c-df448028f65b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:02.733422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770cce7-eb59-42b6-8de8-fd9e7da28f65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:02.734960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770cce7-eb59-42b6-8de8-fd9e7da28f65 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:02.789452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:02.886651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:02.920417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55886b1-96ea-43de-b83c-df448028f65b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:02.954932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 137/266] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:02.956212 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9438f-20a9-4e74-ad2a-140c5ce91af6 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:02.956489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9438f-20a9-4e74-ad2a-140c5ce91af6 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:02.956756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9438f-20a9-4e74-ad2a-140c5ce91af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:02.982736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa1fc-84f9-46b2-932a-95343ee55c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:02.983401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa1fc-84f9-46b2-932a-95343ee55c38 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:02.983630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa1fc-84f9-46b2-932a-95343ee55c38 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:02.983896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa1fc-84f9-46b2-932a-95343ee55c38 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:02.986608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fa1fc-84f9-46b2-932a-95343ee55c38 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:02.988051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fa1fc-84f9-46b2-932a-95343ee55c38 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:02.990508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 130/267] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:02 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:02.994596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-16c9438f-20a9-4e74-ad2a-140c5ce91af6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:02.995477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa1fc-84f9-46b2-932a-95343ee55c38 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:02.997172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-16c9438f-20a9-4e74-ad2a-140c5ce91af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:03.122689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 138/268] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:03.123587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9438f-20a9-4e74-ad2a-140c5ce91af6 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:03.197096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa1fc-84f9-46b2-932a-95343ee55c38 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:03.244104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d93f5-ca05-4818-803a-732d4d282b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:03.244757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d93f5-ca05-4818-803a-732d4d282b5a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:03.245034 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d93f5-ca05-4818-803a-732d4d282b5a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:03.245305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d93f5-ca05-4818-803a-732d4d282b5a None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:03.248656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d93f5-ca05-4818-803a-732d4d282b5a None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:03.250308 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d93f5-ca05-4818-803a-732d4d282b5a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:03.252204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 131/269] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:02 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:03.259027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d93f5-ca05-4818-803a-732d4d282b5a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:03.270934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:03.284232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:03.391124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9438f-20a9-4e74-ad2a-140c5ce91af6 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:03.461042 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d93f5-ca05-4818-803a-732d4d282b5a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:03.465020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92365f49-1221-4bba-83f8-56e3dd93a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:03.465187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92365f49-1221-4bba-83f8-56e3dd93a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:03.465445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92365f49-1221-4bba-83f8-56e3dd93a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:03.465721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92365f49-1221-4bba-83f8-56e3dd93a3e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:03.466459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92365f49-1221-4bba-83f8-56e3dd93a3e2 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=0a33f37b0d0d4ced9878417f46335942, user.id=b678139a68144bebaab2c468d7348a15) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:03.466791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92365f49-1221-4bba-83f8-56e3dd93a3e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:03.468248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92365f49-1221-4bba-83f8-56e3dd93a3e2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:03.475196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 139/270] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:03 2022] GET /identity/v3/role_assignments?user.id=b678139a68144bebaab2c468d7348a15&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=4b10782d99b74f5bb453b82286012e0c => generated 608 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:03.476563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54960c5-db31-4b41-8d62-cc22e52a7fd9 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:03.482932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54960c5-db31-4b41-8d62-cc22e52a7fd9 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:03.482932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54960c5-db31-4b41-8d62-cc22e52a7fd9 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:03.482932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 140/271] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:03.482932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54960c5-db31-4b41-8d62-cc22e52a7fd9 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:03.549966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e57159-c696-422e-b235-8b0eea8f424e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:03.550617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57159-c696-422e-b235-8b0eea8f424e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:03.551451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57159-c696-422e-b235-8b0eea8f424e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:03.551793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57159-c696-422e-b235-8b0eea8f424e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:03.553568 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e57159-c696-422e-b235-8b0eea8f424e None admin] RBAC: Adding query filter params (role.id=c2fe89b1b9fa47b19e02efd64ace9000, scope.project.id=0a33f37b0d0d4ced9878417f46335942, user.id=b678139a68144bebaab2c468d7348a15) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:03.553910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e57159-c696-422e-b235-8b0eea8f424e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:03.559677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e57159-c696-422e-b235-8b0eea8f424e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:03.575137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 132/272] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:03 2022] GET /identity/v3/role_assignments?user.id=b678139a68144bebaab2c468d7348a15&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=c2fe89b1b9fa47b19e02efd64ace9000 => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:03.577166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde3292-6d4f-425c-94da-9119df09111a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:03.577455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde3292-6d4f-425c-94da-9119df09111a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:03.577740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde3292-6d4f-425c-94da-9119df09111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:03.604457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-5fde3292-6d4f-425c-94da-9119df09111a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:03.605702 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5fde3292-6d4f-425c-94da-9119df09111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:03.703274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54960c5-db31-4b41-8d62-cc22e52a7fd9 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:03.728057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:03.729630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 133/273] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:03.740639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94218699-dd8f-4e98-b83a-359e1d3f1470 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:03.741249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94218699-dd8f-4e98-b83a-359e1d3f1470 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:03.741517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94218699-dd8f-4e98-b83a-359e1d3f1470 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:03.743717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 134/274] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:05:03.754026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef188b69-2b01-4ea0-ad85-e96271310cb3 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:03.754285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef188b69-2b01-4ea0-ad85-e96271310cb3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:03.754611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef188b69-2b01-4ea0-ad85-e96271310cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:03.773614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895a897-e2d5-4af8-a9c5-9ce9742ddda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:03.773817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:03.775447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a897-e2d5-4af8-a9c5-9ce9742ddda4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:03.776043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a897-e2d5-4af8-a9c5-9ce9742ddda4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:03.776521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a897-e2d5-4af8-a9c5-9ce9742ddda4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:03.791638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ef188b69-2b01-4ea0-ad85-e96271310cb3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:03.792874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ef188b69-2b01-4ea0-ad85-e96271310cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:03.806780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895a897-e2d5-4af8-a9c5-9ce9742ddda4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-b895a897-e2d5-4af8-a9c5-9ce9742ddda4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:03.815557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:03.817133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:03.817951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:03.819906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 141/275] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:03.831983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895a897-e2d5-4af8-a9c5-9ce9742ddda4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:03.932891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 135/276] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:04.002057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895a897-e2d5-4af8-a9c5-9ce9742ddda4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:04.048725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf63ff-179a-4a60-8631-d337400f54d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:04.049781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf63ff-179a-4a60-8631-d337400f54d2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:04.050261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf63ff-179a-4a60-8631-d337400f54d2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:04.051639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf63ff-179a-4a60-8631-d337400f54d2 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:04.052663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf63ff-179a-4a60-8631-d337400f54d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:04.053229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf63ff-179a-4a60-8631-d337400f54d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:04.054956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf63ff-179a-4a60-8631-d337400f54d2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:04.065194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 142/277] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:04.071054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef188b69-2b01-4ea0-ad85-e96271310cb3 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:04.084331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf63ff-179a-4a60-8631-d337400f54d2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:04.146144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:04.210932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef188b69-2b01-4ea0-ad85-e96271310cb3 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:04.233541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf63ff-179a-4a60-8631-d337400f54d2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:04.260007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2903da0-6c83-4906-a08e-fdac3ceae7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:04.261566 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2903da0-6c83-4906-a08e-fdac3ceae7df None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:04.262022 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2903da0-6c83-4906-a08e-fdac3ceae7df None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:04.262791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2903da0-6c83-4906-a08e-fdac3ceae7df None admin] PATH_INFO: `/v3/users/746cd268c45d46d9a3d0e2b54d52f7a1` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:04.267209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:04.273366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d249ba4-88d2-4beb-8189-26bfcbaebcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:04.274152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d249ba4-88d2-4beb-8189-26bfcbaebcb3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:04.274558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d249ba4-88d2-4beb-8189-26bfcbaebcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:04.274929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d249ba4-88d2-4beb-8189-26bfcbaebcb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:04.276305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d249ba4-88d2-4beb-8189-26bfcbaebcb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:04.277811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d249ba4-88d2-4beb-8189-26bfcbaebcb3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:04.279830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 143/278] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:04.282898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2903da0-6c83-4906-a08e-fdac3ceae7df None admin] RBAC: Authorizing `identity:get_user(user_id=746cd268c45d46d9a3d0e2b54d52f7a1)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:04.283705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d249ba4-88d2-4beb-8189-26bfcbaebcb3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:04.284611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2903da0-6c83-4906-a08e-fdac3ceae7df None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:04.286410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 136/279] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:04 2022] GET /identity/v3/users/746cd268c45d46d9a3d0e2b54d52f7a1 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:04.287513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2903da0-6c83-4906-a08e-fdac3ceae7df None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:04.394408 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2903da0-6c83-4906-a08e-fdac3ceae7df None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:04.396645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d249ba4-88d2-4beb-8189-26bfcbaebcb3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:04.405996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:04.430839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f423bf-d390-482b-a6a1-8aa7e8f926ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:04.431455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f423bf-d390-482b-a6a1-8aa7e8f926ed None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:04.431733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f423bf-d390-482b-a6a1-8aa7e8f926ed None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:04.432002 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f423bf-d390-482b-a6a1-8aa7e8f926ed None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:04.432719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f423bf-d390-482b-a6a1-8aa7e8f926ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:04.433388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba249392-0a8d-4609-9bd8-33d5499376ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:04.434183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba249392-0a8d-4609-9bd8-33d5499376ea None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:04.434286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f423bf-d390-482b-a6a1-8aa7e8f926ed None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:04.434552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba249392-0a8d-4609-9bd8-33d5499376ea None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:04.435091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba249392-0a8d-4609-9bd8-33d5499376ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:04.446651 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba249392-0a8d-4609-9bd8-33d5499376ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ba249392-0a8d-4609-9bd8-33d5499376ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:04.451933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:04.456867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 144/280] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:04.457729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba249392-0a8d-4609-9bd8-33d5499376ea None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:04.537604 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 137/281] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:05:04 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:05:04.538865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283d3d2-896a-4a34-9ab3-b2e06952536e None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:04.539146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283d3d2-896a-4a34-9ab3-b2e06952536e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:04.539421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283d3d2-896a-4a34-9ab3-b2e06952536e None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:04.541105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 138/282] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:04.541708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283d3d2-896a-4a34-9ab3-b2e06952536e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:04.561430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba249392-0a8d-4609-9bd8-33d5499376ea None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:04.605921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596b352-6610-499e-90fc-9e98c5bf797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:04.606504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:04.606622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596b352-6610-499e-90fc-9e98c5bf797d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:04.606844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596b352-6610-499e-90fc-9e98c5bf797d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:04.607123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596b352-6610-499e-90fc-9e98c5bf797d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:04.607873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596b352-6610-499e-90fc-9e98c5bf797d None admin] RBAC: Adding query filter params (role.id=13accf89c12040508974db47c6905b31, user.id=746cd268c45d46d9a3d0e2b54d52f7a1, scope.system=all) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:04.608181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596b352-6610-499e-90fc-9e98c5bf797d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:04.609699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596b352-6610-499e-90fc-9e98c5bf797d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:04.616458 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 145/283] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 22 17:05:04 2022] GET /identity/v3/role_assignments?user.id=746cd268c45d46d9a3d0e2b54d52f7a1&scope.system=all&role.id=13accf89c12040508974db47c6905b31 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:04.618676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fb40b-a326-4d94-a5a7-6d8d4c31aa5d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:04.619384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fb40b-a326-4d94-a5a7-6d8d4c31aa5d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:04.619892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fb40b-a326-4d94-a5a7-6d8d4c31aa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:04.646197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2d8fb40b-a326-4d94-a5a7-6d8d4c31aa5d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:04.647555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2d8fb40b-a326-4d94-a5a7-6d8d4c31aa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:04.686217 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283d3d2-896a-4a34-9ab3-b2e06952536e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:04.690753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:04.739700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012da0f4-dcc0-4207-94be-61ce6ab3ba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:04.741246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da0f4-dcc0-4207-94be-61ce6ab3ba8b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:04.741729 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da0f4-dcc0-4207-94be-61ce6ab3ba8b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:04.742508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da0f4-dcc0-4207-94be-61ce6ab3ba8b None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:04.743405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012da0f4-dcc0-4207-94be-61ce6ab3ba8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:04.743884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012da0f4-dcc0-4207-94be-61ce6ab3ba8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:04.745470 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012da0f4-dcc0-4207-94be-61ce6ab3ba8b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:04.746437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 146/284] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:04.757165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 139/285] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:04.762379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8fb40b-a326-4d94-a5a7-6d8d4c31aa5d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:04.878152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8fb40b-a326-4d94-a5a7-6d8d4c31aa5d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:04.917844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb82554-ba22-41b6-9622-13c20fe19c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:04.918813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82554-ba22-41b6-9622-13c20fe19c5a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:04.919266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82554-ba22-41b6-9622-13c20fe19c5a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:04.919750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82554-ba22-41b6-9622-13c20fe19c5a None admin] PATH_INFO: `/v3/users/44f3b7d8469e4cfaba8ecde8554d9b84` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:04.921123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb82554-ba22-41b6-9622-13c20fe19c5a None admin] RBAC: Authorizing `identity:get_user(user_id=44f3b7d8469e4cfaba8ecde8554d9b84)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:04.923025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb82554-ba22-41b6-9622-13c20fe19c5a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:04.925144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 147/286] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:04 2022] GET /identity/v3/users/44f3b7d8469e4cfaba8ecde8554d9b84 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:04.930934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012da0f4-dcc0-4207-94be-61ce6ab3ba8b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:05.030769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb82554-ba22-41b6-9622-13c20fe19c5a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:05.046894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012da0f4-dcc0-4207-94be-61ce6ab3ba8b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:05.067808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:05.074905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:05.100334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a344e-8e06-4a50-9c3e-bbfda7e3f203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:05.101990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a344e-8e06-4a50-9c3e-bbfda7e3f203 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:05.102521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a344e-8e06-4a50-9c3e-bbfda7e3f203 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:05.103201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a344e-8e06-4a50-9c3e-bbfda7e3f203 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:05.104823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a344e-8e06-4a50-9c3e-bbfda7e3f203 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:05.106302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a344e-8e06-4a50-9c3e-bbfda7e3f203 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:05.108402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 140/287] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:04 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:05.114288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a344e-8e06-4a50-9c3e-bbfda7e3f203 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:05.171542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb82554-ba22-41b6-9622-13c20fe19c5a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:05.212961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2dc09-24af-4d60-b5e5-37ffa83cdc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:05.213865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2dc09-24af-4d60-b5e5-37ffa83cdc8a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:05.214369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2dc09-24af-4d60-b5e5-37ffa83cdc8a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:05.214791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2dc09-24af-4d60-b5e5-37ffa83cdc8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:05.228794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2dc09-24af-4d60-b5e5-37ffa83cdc8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-7be2dc09-24af-4d60-b5e5-37ffa83cdc8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:05.234082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:05.235048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:05.235929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 148/288] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:05.242676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2dc09-24af-4d60-b5e5-37ffa83cdc8a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:05.270449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a344e-8e06-4a50-9c3e-bbfda7e3f203 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:05.353906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5997b-ae15-48de-99ab-c21754e5a2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:05.354766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5997b-ae15-48de-99ab-c21754e5a2a7 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:05.355042 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5997b-ae15-48de-99ab-c21754e5a2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:05.355338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5997b-ae15-48de-99ab-c21754e5a2a7 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942/users/44f3b7d8469e4cfaba8ecde8554d9b84/roles/ef17b1e922a942a083c0f1905c8c523c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:05.363988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5997b-ae15-48de-99ab-c21754e5a2a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a33f37b0d0d4ced9878417f46335942, user_id=44f3b7d8469e4cfaba8ecde8554d9b84, role_id=ef17b1e922a942a083c0f1905c8c523c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:05.365658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5997b-ae15-48de-99ab-c21754e5a2a7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:05.399475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:05.406463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2dc09-24af-4d60-b5e5-37ffa83cdc8a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:05.448439 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdcb99-5b9d-4c79-b492-4f203284b31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:05.448972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdcb99-5b9d-4c79-b492-4f203284b31e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:05.449251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdcb99-5b9d-4c79-b492-4f203284b31e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:05.449520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdcb99-5b9d-4c79-b492-4f203284b31e None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:05.450078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 141/289] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:05:05 2022] PUT /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942/users/44f3b7d8469e4cfaba8ecde8554d9b84/roles/ef17b1e922a942a083c0f1905c8c523c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:05:05.450310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdcb99-5b9d-4c79-b492-4f203284b31e None admin] RBAC: Adding query filter params (name=member) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:05.450569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdcb99-5b9d-4c79-b492-4f203284b31e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:05.452117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdcb99-5b9d-4c79-b492-4f203284b31e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:05.463468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 149/290] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:05.471651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5997b-ae15-48de-99ab-c21754e5a2a7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:05.525028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:05.596864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5997b-ae15-48de-99ab-c21754e5a2a7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:05.638456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28704f4c-76cc-4b0b-b4a1-b82022577ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:05.639132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28704f4c-76cc-4b0b-b4a1-b82022577ffb None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:05.639412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28704f4c-76cc-4b0b-b4a1-b82022577ffb None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:05.639684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28704f4c-76cc-4b0b-b4a1-b82022577ffb None admin] PATH_INFO: `/v3/users/f8ac338975bf4f49ab03504322ee14bf` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:05.640845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28704f4c-76cc-4b0b-b4a1-b82022577ffb None admin] RBAC: Authorizing `identity:get_user(user_id=f8ac338975bf4f49ab03504322ee14bf)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:05.642340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28704f4c-76cc-4b0b-b4a1-b82022577ffb None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:05.644260 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 142/291] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:05 2022] GET /identity/v3/users/f8ac338975bf4f49ab03504322ee14bf => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:05.650177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdcb99-5b9d-4c79-b492-4f203284b31e None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:05.758495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdcb99-5b9d-4c79-b492-4f203284b31e None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:05.794571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917c4fb-410b-4117-88d3-18c0d1d8b344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:05.795186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917c4fb-410b-4117-88d3-18c0d1d8b344 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:05.795462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917c4fb-410b-4117-88d3-18c0d1d8b344 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:05.795773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917c4fb-410b-4117-88d3-18c0d1d8b344 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:05.820253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:05.869500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917c4fb-410b-4117-88d3-18c0d1d8b344 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:05.871102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917c4fb-410b-4117-88d3-18c0d1d8b344 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:05.979207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 150/292] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:05.985053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28704f4c-76cc-4b0b-b4a1-b82022577ffb None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:06.091406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28704f4c-76cc-4b0b-b4a1-b82022577ffb None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:06.101996 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:06.128748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1361f93-70c2-4736-901b-bbb2c5cef319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:06.129655 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1361f93-70c2-4736-901b-bbb2c5cef319 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:06.129935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1361f93-70c2-4736-901b-bbb2c5cef319 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:06.130212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1361f93-70c2-4736-901b-bbb2c5cef319 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:06.134229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1361f93-70c2-4736-901b-bbb2c5cef319 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:06.134229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1361f93-70c2-4736-901b-bbb2c5cef319 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:06.134913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 143/293] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:05 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:06.141007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917c4fb-410b-4117-88d3-18c0d1d8b344 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:06.198702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:06.250739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917c4fb-410b-4117-88d3-18c0d1d8b344 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:06.287420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d3050-8214-4675-9bda-172df817aa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:06.288037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d3050-8214-4675-9bda-172df817aa8f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:06.288312 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d3050-8214-4675-9bda-172df817aa8f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:06.288580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d3050-8214-4675-9bda-172df817aa8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:06.289318 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d3050-8214-4675-9bda-172df817aa8f None admin] RBAC: Adding query filter params (role.id=13accf89c12040508974db47c6905b31, scope.project.id=0a33f37b0d0d4ced9878417f46335942, user.id=f8ac338975bf4f49ab03504322ee14bf) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:06.289622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d3050-8214-4675-9bda-172df817aa8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:06.291107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d3050-8214-4675-9bda-172df817aa8f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:06.297808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 151/294] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:06 2022] GET /identity/v3/role_assignments?user.id=f8ac338975bf4f49ab03504322ee14bf&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=13accf89c12040508974db47c6905b31 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:07.077250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852770-0bfe-420b-807b-263731d80c44 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.077648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852770-0bfe-420b-807b-263731d80c44 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.077822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852770-0bfe-420b-807b-263731d80c44 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.079456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 144/295] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:07.084763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28297ea-6d5e-4b83-bbd3-55440284d7c4 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.085047 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28297ea-6d5e-4b83-bbd3-55440284d7c4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.085325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28297ea-6d5e-4b83-bbd3-55440284d7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.104611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f28297ea-6d5e-4b83-bbd3-55440284d7c4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:07.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f28297ea-6d5e-4b83-bbd3-55440284d7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:07.177544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 152/296] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:07.203020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f0068-48a1-49fa-82c0-ba08e46809dd None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.203983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f0068-48a1-49fa-82c0-ba08e46809dd None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.204448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f0068-48a1-49fa-82c0-ba08e46809dd None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.206324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 145/297] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:07.212599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae192fb-a263-4458-9641-2a2d1ff02ea5 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.212886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae192fb-a263-4458-9641-2a2d1ff02ea5 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.213160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae192fb-a263-4458-9641-2a2d1ff02ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.232315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-dae192fb-a263-4458-9641-2a2d1ff02ea5 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:07.233496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dae192fb-a263-4458-9641-2a2d1ff02ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:07.283365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:07.310053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 153/298] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:07.312981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a4c9b-14be-4577-a4f6-44361163dd88 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.313262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a4c9b-14be-4577-a4f6-44361163dd88 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.313534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a4c9b-14be-4577-a4f6-44361163dd88 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.315093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 146/299] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:07.321595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa751e-a4c0-4aff-972d-10d8d3054ccb None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.322154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa751e-a4c0-4aff-972d-10d8d3054ccb None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.322686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa751e-a4c0-4aff-972d-10d8d3054ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.345693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-10fa751e-a4c0-4aff-972d-10d8d3054ccb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:07.346890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-10fa751e-a4c0-4aff-972d-10d8d3054ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:07.420216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 154/300] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:07.470770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a4c9b-14be-4577-a4f6-44361163dd88 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:07.583531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a4c9b-14be-4577-a4f6-44361163dd88 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:07.600694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa751e-a4c0-4aff-972d-10d8d3054ccb None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:07.633667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d6998-799e-4256-a9cf-cd420954e263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:07.634407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d6998-799e-4256-a9cf-cd420954e263 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.634722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d6998-799e-4256-a9cf-cd420954e263 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.634992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d6998-799e-4256-a9cf-cd420954e263 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.647677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d6998-799e-4256-a9cf-cd420954e263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-8f2d6998-799e-4256-a9cf-cd420954e263 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:07.652482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:07.653400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:07.654215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 147/301] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:07.661654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d6998-799e-4256-a9cf-cd420954e263 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:07.711850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:07.734502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa751e-a4c0-4aff-972d-10d8d3054ccb None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:07.743960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:07.774358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5f3f5-97bb-452d-a8c7-4fba59232570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:07.775026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5f3f5-97bb-452d-a8c7-4fba59232570 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.775303 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5f3f5-97bb-452d-a8c7-4fba59232570 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.775572 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5f3f5-97bb-452d-a8c7-4fba59232570 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.781472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d6998-799e-4256-a9cf-cd420954e263 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:07.786354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5f3f5-97bb-452d-a8c7-4fba59232570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-34d5f3f5-97bb-452d-a8c7-4fba59232570 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:07.791126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:07.792058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:07.792855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 155/302] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:07.793657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5f3f5-97bb-452d-a8c7-4fba59232570 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:07.818040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52805b8b-055e-4cf9-ad8b-6f32454e7b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:07.818680 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52805b8b-055e-4cf9-ad8b-6f32454e7b29 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.818959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52805b8b-055e-4cf9-ad8b-6f32454e7b29 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.819230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52805b8b-055e-4cf9-ad8b-6f32454e7b29 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.819901 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52805b8b-055e-4cf9-ad8b-6f32454e7b29 None admin] RBAC: Adding query filter params (name=member) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:07.820202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52805b8b-055e-4cf9-ad8b-6f32454e7b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:07.821582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52805b8b-055e-4cf9-ad8b-6f32454e7b29 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:07.835574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 148/303] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:07.835574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77896-62f2-4c53-89e8-44f4d93b4324 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.835574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77896-62f2-4c53-89e8-44f4d93b4324 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.835574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77896-62f2-4c53-89e8-44f4d93b4324 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.835574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 149/304] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:07.835574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77896-62f2-4c53-89e8-44f4d93b4324 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:07.896289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5f3f5-97bb-452d-a8c7-4fba59232570 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:07.930473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54784875-3373-4ba8-8881-7363a810bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:07.931366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54784875-3373-4ba8-8881-7363a810bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.931668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54784875-3373-4ba8-8881-7363a810bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.931946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54784875-3373-4ba8-8881-7363a810bbd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.933119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54784875-3373-4ba8-8881-7363a810bbd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:07.934654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54784875-3373-4ba8-8881-7363a810bbd0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:07.936773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 156/305] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:07.937918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981edd18-3d29-4dd5-b0c1-3ce4704e75df None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.938220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981edd18-3d29-4dd5-b0c1-3ce4704e75df None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.938539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981edd18-3d29-4dd5-b0c1-3ce4704e75df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.939641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77896-62f2-4c53-89e8-44f4d93b4324 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:07.957570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-981edd18-3d29-4dd5-b0c1-3ce4704e75df None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:07.959030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-981edd18-3d29-4dd5-b0c1-3ce4704e75df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:07.975397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f3ee5-e0a7-404c-91b1-5f2f82871c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:07.976006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f3ee5-e0a7-404c-91b1-5f2f82871c9a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:07.976282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f3ee5-e0a7-404c-91b1-5f2f82871c9a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:07.976552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f3ee5-e0a7-404c-91b1-5f2f82871c9a None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:07.977219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f3ee5-e0a7-404c-91b1-5f2f82871c9a None admin] RBAC: Adding query filter params (name=reader) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:07.977524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f3ee5-e0a7-404c-91b1-5f2f82871c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:07.978954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f3ee5-e0a7-404c-91b1-5f2f82871c9a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:07.987445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 150/306] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:07.988142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f3ee5-e0a7-404c-91b1-5f2f82871c9a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:08.031608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 157/307] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:08.032457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981edd18-3d29-4dd5-b0c1-3ce4704e75df None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:08.042328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:08.081939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:08.101439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f3ee5-e0a7-404c-91b1-5f2f82871c9a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:08.148324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef7adc-3517-4034-954d-88ec5a83aea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:08.148876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef7adc-3517-4034-954d-88ec5a83aea9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:08.149146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef7adc-3517-4034-954d-88ec5a83aea9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:08.149601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef7adc-3517-4034-954d-88ec5a83aea9 None admin] PATH_INFO: `/v3/users/746cd268c45d46d9a3d0e2b54d52f7a1` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:08.150558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef7adc-3517-4034-954d-88ec5a83aea9 None admin] RBAC: Authorizing `identity:get_user(user_id=746cd268c45d46d9a3d0e2b54d52f7a1)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:08.152013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef7adc-3517-4034-954d-88ec5a83aea9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:08.152450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981edd18-3d29-4dd5-b0c1-3ce4704e75df None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:08.158057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 151/308] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:07 2022] GET /identity/v3/users/746cd268c45d46d9a3d0e2b54d52f7a1 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:08.158057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef7adc-3517-4034-954d-88ec5a83aea9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:08.190964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:08.198707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970c0fc-ae71-4d49-b253-c2ac780450bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:08.202832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970c0fc-ae71-4d49-b253-c2ac780450bf None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:08.202832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970c0fc-ae71-4d49-b253-c2ac780450bf None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:08.202832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970c0fc-ae71-4d49-b253-c2ac780450bf None admin] PATH_INFO: `/v3/groups` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:08.202832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970c0fc-ae71-4d49-b253-c2ac780450bf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:08.202832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970c0fc-ae71-4d49-b253-c2ac780450bf None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:08.272849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef7adc-3517-4034-954d-88ec5a83aea9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:08.309150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a71691-41ed-4f81-9138-7c6493b8ca26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:08.309781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a71691-41ed-4f81-9138-7c6493b8ca26 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:08.310058 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a71691-41ed-4f81-9138-7c6493b8ca26 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:08.310327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a71691-41ed-4f81-9138-7c6493b8ca26 None admin] PATH_INFO: `/v3/users/97ddd54bed064a388e7af44daf453763` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:08.324057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a71691-41ed-4f81-9138-7c6493b8ca26 None admin] RBAC: Authorizing `identity:get_user(user_id=97ddd54bed064a388e7af44daf453763)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:08.325698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a71691-41ed-4f81-9138-7c6493b8ca26 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:08.327462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 152/309] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:08 2022] GET /identity/v3/users/97ddd54bed064a388e7af44daf453763 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:08.328296 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a71691-41ed-4f81-9138-7c6493b8ca26 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:08.412129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:08.426981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a71691-41ed-4f81-9138-7c6493b8ca26 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:08.459313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f42aa-4cdf-4c73-b385-585f6e349104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:08.459957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f42aa-4cdf-4c73-b385-585f6e349104 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:08.460233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f42aa-4cdf-4c73-b385-585f6e349104 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:08.460504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f42aa-4cdf-4c73-b385-585f6e349104 None admin] PATH_INFO: `/v3/system/users/746cd268c45d46d9a3d0e2b54d52f7a1/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:08.461886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f42aa-4cdf-4c73-b385-585f6e349104 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=746cd268c45d46d9a3d0e2b54d52f7a1, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:08.463300 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f42aa-4cdf-4c73-b385-585f6e349104 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:08.525847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 158/310] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Aug 22 17:05:08 2022] POST /identity/v3/groups => generated 229 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:05:08.544498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970c0fc-ae71-4d49-b253-c2ac780450bf None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:08.555565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:08.644125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970c0fc-ae71-4d49-b253-c2ac780450bf None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:08.675889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 153/311] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Aug 22 17:05:08 2022] PUT /identity/v3/system/users/746cd268c45d46d9a3d0e2b54d52f7a1/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:05:08.675889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f42aa-4cdf-4c73-b385-585f6e349104 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:08.685963 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99982b-c272-46f4-9bc6-3a1991fd8754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:08.686680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99982b-c272-46f4-9bc6-3a1991fd8754 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:08.687015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99982b-c272-46f4-9bc6-3a1991fd8754 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:08.687237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99982b-c272-46f4-9bc6-3a1991fd8754 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:08.699949 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99982b-c272-46f4-9bc6-3a1991fd8754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-0e99982b-c272-46f4-9bc6-3a1991fd8754 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:08.705175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:08.706552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:08.707942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 159/312] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:08.714730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99982b-c272-46f4-9bc6-3a1991fd8754 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:08.846453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:08.868039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99982b-c272-46f4-9bc6-3a1991fd8754 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:08.888971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f42aa-4cdf-4c73-b385-585f6e349104 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:08.919892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8e6de-b55a-42b3-86cc-e1141ec16745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:08.921167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8e6de-b55a-42b3-86cc-e1141ec16745 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:08.921828 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8e6de-b55a-42b3-86cc-e1141ec16745 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:08.922249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8e6de-b55a-42b3-86cc-e1141ec16745 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:08.923601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8e6de-b55a-42b3-86cc-e1141ec16745 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:08.924539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8e6de-b55a-42b3-86cc-e1141ec16745 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:08.926543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8e6de-b55a-42b3-86cc-e1141ec16745 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:08.930624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053605c-b85f-4705-b8fb-0bf47abc198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:08.931420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053605c-b85f-4705-b8fb-0bf47abc198d None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:08.931801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053605c-b85f-4705-b8fb-0bf47abc198d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:08.932165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053605c-b85f-4705-b8fb-0bf47abc198d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:08.933189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053605c-b85f-4705-b8fb-0bf47abc198d None admin] RBAC: Adding query filter params (role.id=ef17b1e922a942a083c0f1905c8c523c, user.id=97ddd54bed064a388e7af44daf453763, scope.system=all) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:08.933619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053605c-b85f-4705-b8fb-0bf47abc198d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:08.935232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053605c-b85f-4705-b8fb-0bf47abc198d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:08.937819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 160/313] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:08.944834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8e6de-b55a-42b3-86cc-e1141ec16745 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:08.946348 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 154/314] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 22 17:05:08 2022] GET /identity/v3/role_assignments?user.id=97ddd54bed064a388e7af44daf453763&scope.system=all&role.id=ef17b1e922a942a083c0f1905c8c523c => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:09.062001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8e6de-b55a-42b3-86cc-e1141ec16745 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:09.106796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758ad82-5834-4bdc-a7bd-863cda393097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:09.107451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758ad82-5834-4bdc-a7bd-863cda393097 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:09.107727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758ad82-5834-4bdc-a7bd-863cda393097 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:09.107999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758ad82-5834-4bdc-a7bd-863cda393097 None admin] PATH_INFO: `/v3/users/44f3b7d8469e4cfaba8ecde8554d9b84` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:09.109322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758ad82-5834-4bdc-a7bd-863cda393097 None admin] RBAC: Authorizing `identity:get_user(user_id=44f3b7d8469e4cfaba8ecde8554d9b84)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:09.110825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758ad82-5834-4bdc-a7bd-863cda393097 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:09.112779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 161/315] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:08 2022] GET /identity/v3/users/44f3b7d8469e4cfaba8ecde8554d9b84 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:09.118761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053605c-b85f-4705-b8fb-0bf47abc198d None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:09.137802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:09.234052 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053605c-b85f-4705-b8fb-0bf47abc198d None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:09.271354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8742e-ad7a-4048-aa90-4f80583891f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:09.271969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8742e-ad7a-4048-aa90-4f80583891f2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:09.272239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8742e-ad7a-4048-aa90-4f80583891f2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:09.272504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8742e-ad7a-4048-aa90-4f80583891f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:09.349650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8742e-ad7a-4048-aa90-4f80583891f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:09.351799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8742e-ad7a-4048-aa90-4f80583891f2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:09.368016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:09.456522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 155/316] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:09.462619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758ad82-5834-4bdc-a7bd-863cda393097 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:09.481386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:09.574416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758ad82-5834-4bdc-a7bd-863cda393097 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:09.623226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2b451-3c3b-4ead-ae96-17e33d882e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:09.624182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2b451-3c3b-4ead-ae96-17e33d882e61 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:09.624182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2b451-3c3b-4ead-ae96-17e33d882e61 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:09.626543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2b451-3c3b-4ead-ae96-17e33d882e61 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:09.626543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2b451-3c3b-4ead-ae96-17e33d882e61 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:09.627321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2b451-3c3b-4ead-ae96-17e33d882e61 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:09.630874 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 162/317] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:09 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:09.636455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8742e-ad7a-4048-aa90-4f80583891f2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:09.755907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8742e-ad7a-4048-aa90-4f80583891f2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:09.773681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:09.798921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba916aad-a650-42cb-80f4-b0a419d381d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:09.799583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba916aad-a650-42cb-80f4-b0a419d381d9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:09.799886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba916aad-a650-42cb-80f4-b0a419d381d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:09.800188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba916aad-a650-42cb-80f4-b0a419d381d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:09.801179 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba916aad-a650-42cb-80f4-b0a419d381d9 None admin] RBAC: Adding query filter params (role.id=ef17b1e922a942a083c0f1905c8c523c, scope.project.id=0a33f37b0d0d4ced9878417f46335942, user.id=44f3b7d8469e4cfaba8ecde8554d9b84) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:09.801638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba916aad-a650-42cb-80f4-b0a419d381d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:09.803185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba916aad-a650-42cb-80f4-b0a419d381d9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:09.811244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 156/318] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:09 2022] GET /identity/v3/role_assignments?user.id=44f3b7d8469e4cfaba8ecde8554d9b84&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=ef17b1e922a942a083c0f1905c8c523c => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:10.584641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaee5a-66d5-4c70-b20a-e8094691753f None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:10.585171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaee5a-66d5-4c70-b20a-e8094691753f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:10.585171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaee5a-66d5-4c70-b20a-e8094691753f None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:10.589714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 163/319] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:10.592384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a25860-c61e-44ea-a30c-545695269b35 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:10.592795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a25860-c61e-44ea-a30c-545695269b35 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:10.592978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a25860-c61e-44ea-a30c-545695269b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:10.612242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-90a25860-c61e-44ea-a30c-545695269b35 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:10.613520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-90a25860-c61e-44ea-a30c-545695269b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:10.690530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 157/320] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:10.799395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d14758-7959-4e39-be51-4fb55717ed91 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:10.799717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d14758-7959-4e39-be51-4fb55717ed91 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:10.799931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d14758-7959-4e39-be51-4fb55717ed91 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:10.802307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 164/321] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:10.807707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af900703-93c1-489e-aca3-0ee37e2c409b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:10.807993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af900703-93c1-489e-aca3-0ee37e2c409b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:10.808261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af900703-93c1-489e-aca3-0ee37e2c409b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:10.827381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-af900703-93c1-489e-aca3-0ee37e2c409b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:10.828575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-af900703-93c1-489e-aca3-0ee37e2c409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:10.900019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 158/322] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:10.951245 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87be165-f260-490b-aa0f-086550fa390b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:10.951607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87be165-f260-490b-aa0f-086550fa390b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:10.951946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87be165-f260-490b-aa0f-086550fa390b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:10.953376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 165/323] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:10.959489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25285e8-2939-4915-9b93-07efbc39c0f3 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:10.959778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25285e8-2939-4915-9b93-07efbc39c0f3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:10.960056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25285e8-2939-4915-9b93-07efbc39c0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:10.978640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d25285e8-2939-4915-9b93-07efbc39c0f3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:10.979541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87be165-f260-490b-aa0f-086550fa390b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:10.979848 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d25285e8-2939-4915-9b93-07efbc39c0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:11.014104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:11.061237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 159/324] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:11.078289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:11.099221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87be165-f260-490b-aa0f-086550fa390b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:11.137736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3b002-c401-4e53-a9c7-b357fa553ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:11.138361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3b002-c401-4e53-a9c7-b357fa553ebd None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:11.138661 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3b002-c401-4e53-a9c7-b357fa553ebd None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:11.138926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3b002-c401-4e53-a9c7-b357fa553ebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:11.140141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3b002-c401-4e53-a9c7-b357fa553ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:11.141478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3b002-c401-4e53-a9c7-b357fa553ebd None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:11.143355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 166/325] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:11.149657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25285e8-2939-4915-9b93-07efbc39c0f3 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:11.178403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3b002-c401-4e53-a9c7-b357fa553ebd None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:11.292777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25285e8-2939-4915-9b93-07efbc39c0f3 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:11.303722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3b002-c401-4e53-a9c7-b357fa553ebd None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:11.336235 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb087ed-7841-459d-87a7-b3bfad288ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:11.344789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb087ed-7841-459d-87a7-b3bfad288ea9 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:11.344789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb087ed-7841-459d-87a7-b3bfad288ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:11.344789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb087ed-7841-459d-87a7-b3bfad288ea9 None admin] PATH_INFO: `/v3/groups` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:11.344789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb087ed-7841-459d-87a7-b3bfad288ea9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:11.344789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb087ed-7841-459d-87a7-b3bfad288ea9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:11.355037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab6c72-1ef3-42f0-b64a-9599d8fe40a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:11.355669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab6c72-1ef3-42f0-b64a-9599d8fe40a2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:11.355939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab6c72-1ef3-42f0-b64a-9599d8fe40a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:11.356207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab6c72-1ef3-42f0-b64a-9599d8fe40a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:11.366177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab6c72-1ef3-42f0-b64a-9599d8fe40a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-7bab6c72-1ef3-42f0-b64a-9599d8fe40a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:11.370825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:11.371880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 167/326] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:11.372676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab6c72-1ef3-42f0-b64a-9599d8fe40a2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:11.385718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:11.473689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 160/327] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Aug 22 17:05:11 2022] POST /identity/v3/groups => generated 226 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:05:11.474150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab6c72-1ef3-42f0-b64a-9599d8fe40a2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:11.491685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb087ed-7841-459d-87a7-b3bfad288ea9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:11.523470 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f67c6-c449-4eae-bda5-6ec17fd36bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:11.524601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f67c6-c449-4eae-bda5-6ec17fd36bc9 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:11.525113 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f67c6-c449-4eae-bda5-6ec17fd36bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:11.526310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f67c6-c449-4eae-bda5-6ec17fd36bc9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:11.533484 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:11.544117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f67c6-c449-4eae-bda5-6ec17fd36bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-9f7f67c6-c449-4eae-bda5-6ec17fd36bc9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:11.549706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:11.552073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:11.553645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 168/328] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:11.558440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f67c6-c449-4eae-bda5-6ec17fd36bc9 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:11.632855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb087ed-7841-459d-87a7-b3bfad288ea9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:11.650588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:11.680550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48624aee-3c01-4f0e-902f-5d01f00c41a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:11.681247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48624aee-3c01-4f0e-902f-5d01f00c41a7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:11.681931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48624aee-3c01-4f0e-902f-5d01f00c41a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:11.682149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48624aee-3c01-4f0e-902f-5d01f00c41a7 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:11.683041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48624aee-3c01-4f0e-902f-5d01f00c41a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:11.683371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48624aee-3c01-4f0e-902f-5d01f00c41a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:11.684867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48624aee-3c01-4f0e-902f-5d01f00c41a7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:11.692231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f67c6-c449-4eae-bda5-6ec17fd36bc9 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:11.695946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 161/329] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:11.701796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48624aee-3c01-4f0e-902f-5d01f00c41a7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:11.731568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9eee9-50f1-4862-9f4f-46f39c16ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:11.732114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9eee9-50f1-4862-9f4f-46f39c16ba6e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:11.732389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9eee9-50f1-4862-9f4f-46f39c16ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:11.733078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9eee9-50f1-4862-9f4f-46f39c16ba6e None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:11.733335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9eee9-50f1-4862-9f4f-46f39c16ba6e None admin] RBAC: Adding query filter params (name=reader) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:11.733776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9eee9-50f1-4862-9f4f-46f39c16ba6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:11.735204 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9eee9-50f1-4862-9f4f-46f39c16ba6e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:11.746572 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 169/330] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:11.752849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9eee9-50f1-4862-9f4f-46f39c16ba6e None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:11.836474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:11.850248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48624aee-3c01-4f0e-902f-5d01f00c41a7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:11.893421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9eee9-50f1-4862-9f4f-46f39c16ba6e None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:11.897053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396b2b6-51cb-42d2-a917-fb642073c315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:11.897690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b2b6-51cb-42d2-a917-fb642073c315 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:11.897959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b2b6-51cb-42d2-a917-fb642073c315 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:11.898224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b2b6-51cb-42d2-a917-fb642073c315 None admin] PATH_INFO: `/v3/users/746cd268c45d46d9a3d0e2b54d52f7a1` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:11.899421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396b2b6-51cb-42d2-a917-fb642073c315 None admin] RBAC: Authorizing `identity:get_user(user_id=746cd268c45d46d9a3d0e2b54d52f7a1)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:11.900867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396b2b6-51cb-42d2-a917-fb642073c315 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:11.902789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 162/331] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:11 2022] GET /identity/v3/users/746cd268c45d46d9a3d0e2b54d52f7a1 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:11.908559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396b2b6-51cb-42d2-a917-fb642073c315 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:11.926747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:11.932242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628aa835-80b1-4afb-9f4f-91065923eca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:11.933144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aa835-80b1-4afb-9f4f-91065923eca3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:11.933425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aa835-80b1-4afb-9f4f-91065923eca3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:11.933691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aa835-80b1-4afb-9f4f-91065923eca3 None admin] PATH_INFO: `/v3/users/97ddd54bed064a388e7af44daf453763` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:11.935685 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628aa835-80b1-4afb-9f4f-91065923eca3 None admin] RBAC: Authorizing `identity:get_user(user_id=97ddd54bed064a388e7af44daf453763)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:11.936412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628aa835-80b1-4afb-9f4f-91065923eca3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:11.939637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 170/332] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:11 2022] GET /identity/v3/users/97ddd54bed064a388e7af44daf453763 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:11.946501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628aa835-80b1-4afb-9f4f-91065923eca3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:12.038322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396b2b6-51cb-42d2-a917-fb642073c315 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:12.079430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628aa835-80b1-4afb-9f4f-91065923eca3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:12.088363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3db406-8310-4832-bd9b-bb4a9ff35803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:12.088982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3db406-8310-4832-bd9b-bb4a9ff35803 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:12.089261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3db406-8310-4832-bd9b-bb4a9ff35803 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:12.089526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3db406-8310-4832-bd9b-bb4a9ff35803 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:12.090258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3db406-8310-4832-bd9b-bb4a9ff35803 None admin] RBAC: Adding query filter params (role.id=13accf89c12040508974db47c6905b31, user.id=746cd268c45d46d9a3d0e2b54d52f7a1, scope.system=all) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:12.091091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3db406-8310-4832-bd9b-bb4a9ff35803 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:12.093353 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3db406-8310-4832-bd9b-bb4a9ff35803 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:12.114134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 163/333] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 22 17:05:11 2022] GET /identity/v3/role_assignments?user.id=746cd268c45d46d9a3d0e2b54d52f7a1&scope.system=all&role.id=13accf89c12040508974db47c6905b31 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:12.125212 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182729-ce65-4cb3-b272-d7bf9ebaa88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:12.125901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182729-ce65-4cb3-b272-d7bf9ebaa88d None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:12.126177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182729-ce65-4cb3-b272-d7bf9ebaa88d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:12.126452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182729-ce65-4cb3-b272-d7bf9ebaa88d None admin] PATH_INFO: `/v3/system/users/97ddd54bed064a388e7af44daf453763/roles/ef17b1e922a942a083c0f1905c8c523c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:12.128052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18182729-ce65-4cb3-b272-d7bf9ebaa88d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=97ddd54bed064a388e7af44daf453763, role_id=ef17b1e922a942a083c0f1905c8c523c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:12.129716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18182729-ce65-4cb3-b272-d7bf9ebaa88d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:12.217371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 171/334] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Aug 22 17:05:11 2022] PUT /identity/v3/system/users/97ddd54bed064a388e7af44daf453763/roles/ef17b1e922a942a083c0f1905c8c523c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:05:13.881721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92e471-f345-4a62-b309-98a4c8179d43 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:13.882244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92e471-f345-4a62-b309-98a4c8179d43 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:13.882244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92e471-f345-4a62-b309-98a4c8179d43 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:13.883935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 164/335] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:13.889167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cd99a-7bb7-4dbe-b85d-6abd6892cc36 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:13.889607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cd99a-7bb7-4dbe-b85d-6abd6892cc36 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:13.889783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cd99a-7bb7-4dbe-b85d-6abd6892cc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:13.910938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e96cd99a-7bb7-4dbe-b85d-6abd6892cc36 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:13.912143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e96cd99a-7bb7-4dbe-b85d-6abd6892cc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:13.983760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:13.993026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 172/336] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:14.394125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92e471-f345-4a62-b309-98a4c8179d43 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:14.415504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:14.507713 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acb109-27c0-4ad2-b213-a4355cda00f3 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:14.507991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acb109-27c0-4ad2-b213-a4355cda00f3 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:14.508266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acb109-27c0-4ad2-b213-a4355cda00f3 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:14.509654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 173/337] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:05:14.515423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b85b-d925-4505-bb2d-219dcafbf25e None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:14.515706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b85b-d925-4505-bb2d-219dcafbf25e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:14.515969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b85b-d925-4505-bb2d-219dcafbf25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:14.526375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92e471-f345-4a62-b309-98a4c8179d43 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:14.551761 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-eb96b85b-d925-4505-bb2d-219dcafbf25e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:14.552166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-eb96b85b-d925-4505-bb2d-219dcafbf25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:14.578875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53261a-43b9-4960-bda5-7d2615dcf01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:14.579837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53261a-43b9-4960-bda5-7d2615dcf01b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:14.582883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53261a-43b9-4960-bda5-7d2615dcf01b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:14.582883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53261a-43b9-4960-bda5-7d2615dcf01b None admin] PATH_INFO: `/v3/roles/member` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:14.594544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53261a-43b9-4960-bda5-7d2615dcf01b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-1e53261a-43b9-4960-bda5-7d2615dcf01b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:14.600912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:14.601828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:14.602643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:14.605390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 165/338] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:14.634605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53261a-43b9-4960-bda5-7d2615dcf01b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:14.647893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 174/339] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:14.749028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:14.778309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53261a-43b9-4960-bda5-7d2615dcf01b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:14.822146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd82d7e-b7d5-4484-a6cd-a98a1ae2f5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:14.822786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd82d7e-b7d5-4484-a6cd-a98a1ae2f5e0 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:14.823061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd82d7e-b7d5-4484-a6cd-a98a1ae2f5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:14.823328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd82d7e-b7d5-4484-a6cd-a98a1ae2f5e0 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:14.824269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd82d7e-b7d5-4484-a6cd-a98a1ae2f5e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:14.824571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd82d7e-b7d5-4484-a6cd-a98a1ae2f5e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:14.825948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd82d7e-b7d5-4484-a6cd-a98a1ae2f5e0 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:14.835377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 166/340] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:14.841480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b85b-d925-4505-bb2d-219dcafbf25e None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:14.963226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b85b-d925-4505-bb2d-219dcafbf25e None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:14.989321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:15.000127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24020c1b-7b75-4f24-b3d3-2639e5db65de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:15.000795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24020c1b-7b75-4f24-b3d3-2639e5db65de None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.001073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24020c1b-7b75-4f24-b3d3-2639e5db65de None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.001336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24020c1b-7b75-4f24-b3d3-2639e5db65de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:15.015076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249267ec-fe23-4135-ac4b-f0b492708b4b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.016004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249267ec-fe23-4135-ac4b-f0b492708b4b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.016468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249267ec-fe23-4135-ac4b-f0b492708b4b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:15.020998 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 167/341] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:15.027263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f693b-9ac5-4cc3-aa9a-675fde935c40 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.027894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f693b-9ac5-4cc3-aa9a-675fde935c40 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.028321 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f693b-9ac5-4cc3-aa9a-675fde935c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:15.052357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-0b6f693b-9ac5-4cc3-aa9a-675fde935c40 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:15.054082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0b6f693b-9ac5-4cc3-aa9a-675fde935c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:15.082804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24020c1b-7b75-4f24-b3d3-2639e5db65de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:15.084287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24020c1b-7b75-4f24-b3d3-2639e5db65de None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:15.146649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 168/342] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:15.147982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f693b-9ac5-4cc3-aa9a-675fde935c40 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:15.190281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 175/343] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:15.192100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c680f2c-e52c-499d-9277-d604d0a9c728 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.192484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c680f2c-e52c-499d-9277-d604d0a9c728 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.192764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c680f2c-e52c-499d-9277-d604d0a9c728 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:15.194186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 176/344] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:15.200254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b46342-f7d1-4264-8319-54955c89072b None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.200537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b46342-f7d1-4264-8319-54955c89072b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.200939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b46342-f7d1-4264-8319-54955c89072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:15.221256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f0b46342-f7d1-4264-8319-54955c89072b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:15.222558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f0b46342-f7d1-4264-8319-54955c89072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:15.252475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:15.262942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:15.298010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f693b-9ac5-4cc3-aa9a-675fde935c40 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:15.299073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 177/345] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:15.299984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b46342-f7d1-4264-8319-54955c89072b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:15.339496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb299c9-dbb9-4d16-8560-68a202b4c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:15.340511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb299c9-dbb9-4d16-8560-68a202b4c99c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.340980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb299c9-dbb9-4d16-8560-68a202b4c99c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.341440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb299c9-dbb9-4d16-8560-68a202b4c99c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:15.355415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb299c9-dbb9-4d16-8560-68a202b4c99c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-cfb299c9-dbb9-4d16-8560-68a202b4c99c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:15.361832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:15.363583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:15.364513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:15.367007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 169/346] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:05:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:15.373436 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb299c9-dbb9-4d16-8560-68a202b4c99c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:15.416944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b46342-f7d1-4264-8319-54955c89072b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:15.458256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16a2a5-da77-46fb-9b94-1e6a02df4321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:15.458931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16a2a5-da77-46fb-9b94-1e6a02df4321 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.459201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16a2a5-da77-46fb-9b94-1e6a02df4321 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.459467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16a2a5-da77-46fb-9b94-1e6a02df4321 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:15.460604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16a2a5-da77-46fb-9b94-1e6a02df4321 None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:15.462224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16a2a5-da77-46fb-9b94-1e6a02df4321 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:15.463915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 178/347] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:15 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:15.469588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16a2a5-da77-46fb-9b94-1e6a02df4321 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:15.519035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:15.555799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb299c9-dbb9-4d16-8560-68a202b4c99c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:15.590715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16a2a5-da77-46fb-9b94-1e6a02df4321 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:15.592920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:15.609282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bdb8b-d821-4f06-b494-bc85b56da3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:15.609904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bdb8b-d821-4f06-b494-bc85b56da3a2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.610184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bdb8b-d821-4f06-b494-bc85b56da3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.610450 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bdb8b-d821-4f06-b494-bc85b56da3a2 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:15.611152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34bdb8b-d821-4f06-b494-bc85b56da3a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:15.611694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34bdb8b-d821-4f06-b494-bc85b56da3a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:15.613364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34bdb8b-d821-4f06-b494-bc85b56da3a2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:15.637453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 170/348] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:15.639907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9c818-6119-459f-9a30-bf1d52aed8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:15.640740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9c818-6119-459f-9a30-bf1d52aed8b4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.641190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9c818-6119-459f-9a30-bf1d52aed8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.641674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9c818-6119-459f-9a30-bf1d52aed8b4 None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:15.642745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bdb8b-d821-4f06-b494-bc85b56da3a2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:15.661722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9c818-6119-459f-9a30-bf1d52aed8b4 None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:15.663493 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9c818-6119-459f-9a30-bf1d52aed8b4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:15.667535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 179/349] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:15 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:15.683046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9c818-6119-459f-9a30-bf1d52aed8b4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:15.814891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:15.836494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9c818-6119-459f-9a30-bf1d52aed8b4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:15.849406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bdb8b-d821-4f06-b494-bc85b56da3a2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:15.876168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0d33-a284-46a1-a49e-2f330c2e6d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:15.877009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0d33-a284-46a1-a49e-2f330c2e6d1c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.877432 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0d33-a284-46a1-a49e-2f330c2e6d1c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.877824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0d33-a284-46a1-a49e-2f330c2e6d1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:15.879251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e0d33-a284-46a1-a49e-2f330c2e6d1c None admin] RBAC: Adding query filter params (group.id=3348dd190d744e649fe289aaee2c6877, role.id=13accf89c12040508974db47c6905b31, scope.project.id=c84516b7267f4b43bf816381e3e60ae6) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:15.879718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e0d33-a284-46a1-a49e-2f330c2e6d1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:15.881837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e0d33-a284-46a1-a49e-2f330c2e6d1c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:15.889148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 180/350] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 22 17:05:15 2022] GET /identity/v3/role_assignments?group.id=3348dd190d744e649fe289aaee2c6877&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=13accf89c12040508974db47c6905b31 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:15.890036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0d33-a284-46a1-a49e-2f330c2e6d1c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:15.910513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa071e-aa23-4bfe-8070-2629adc3f595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:15.910513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa071e-aa23-4bfe-8070-2629adc3f595 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:15.910513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa071e-aa23-4bfe-8070-2629adc3f595 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:15.910513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa071e-aa23-4bfe-8070-2629adc3f595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.002980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:16.019653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa071e-aa23-4bfe-8070-2629adc3f595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:16.021406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa071e-aa23-4bfe-8070-2629adc3f595 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:16.059334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0d33-a284-46a1-a49e-2f330c2e6d1c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:16.118551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066219c-d436-47e2-bc1a-36b77fae6d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:16.119382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066219c-d436-47e2-bc1a-36b77fae6d69 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:16.119664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066219c-d436-47e2-bc1a-36b77fae6d69 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:16.119942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066219c-d436-47e2-bc1a-36b77fae6d69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.137774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066219c-d436-47e2-bc1a-36b77fae6d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-1066219c-d436-47e2-bc1a-36b77fae6d69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:16.145500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:16.146678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 181/351] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:16.148029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066219c-d436-47e2-bc1a-36b77fae6d69 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:16.160946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 171/352] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:16.162449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23520-88f6-483b-b0dd-715c8a9358f1 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:16.162779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23520-88f6-483b-b0dd-715c8a9358f1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:16.163048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23520-88f6-483b-b0dd-715c8a9358f1 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.164689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 172/353] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:16.165824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d23520-88f6-483b-b0dd-715c8a9358f1 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:16.173189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:16.288588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:16.304773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d23520-88f6-483b-b0dd-715c8a9358f1 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:16.308740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066219c-d436-47e2-bc1a-36b77fae6d69 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:16.350381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e8e1c-3f54-4225-9190-6492212fafff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:16.351057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e8e1c-3f54-4225-9190-6492212fafff None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:16.351383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e8e1c-3f54-4225-9190-6492212fafff None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:16.351658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e8e1c-3f54-4225-9190-6492212fafff None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.352338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e8e1c-3f54-4225-9190-6492212fafff None admin] RBAC: Adding query filter params (name=reader) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:16.352785 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e8e1c-3f54-4225-9190-6492212fafff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:16.355662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045145-e6b7-4e31-bbeb-4be0163f1018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:16.355958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e8e1c-3f54-4225-9190-6492212fafff None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:16.356324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045145-e6b7-4e31-bbeb-4be0163f1018 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:16.356604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045145-e6b7-4e31-bbeb-4be0163f1018 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:16.356878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045145-e6b7-4e31-bbeb-4be0163f1018 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.366585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 173/354] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:16.367550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e8e1c-3f54-4225-9190-6492212fafff None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:16.371837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab045145-e6b7-4e31-bbeb-4be0163f1018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ab045145-e6b7-4e31-bbeb-4be0163f1018 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:16.378771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:16.381156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 182/355] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:16.383141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c348c-b72c-4913-bb4d-6df3042b0b3f None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:16.384833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c348c-b72c-4913-bb4d-6df3042b0b3f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:16.384833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c348c-b72c-4913-bb4d-6df3042b0b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.413475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-100c348c-b72c-4913-bb4d-6df3042b0b3f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:16.414851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-100c348c-b72c-4913-bb4d-6df3042b0b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:16.500104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:16.505377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 183/356] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:16.506050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c348c-b72c-4913-bb4d-6df3042b0b3f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:16.517311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e8e1c-3f54-4225-9190-6492212fafff None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:16.558762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301238e-6105-4414-b6f5-f5788ce5408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:16.559887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301238e-6105-4414-b6f5-f5788ce5408f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:16.560386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301238e-6105-4414-b6f5-f5788ce5408f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:16.560860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301238e-6105-4414-b6f5-f5788ce5408f None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.562588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301238e-6105-4414-b6f5-f5788ce5408f None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:16.564956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301238e-6105-4414-b6f5-f5788ce5408f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:16.569225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 174/357] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:16 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:16.570358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301238e-6105-4414-b6f5-f5788ce5408f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:16.634900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:16.635079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c348c-b72c-4913-bb4d-6df3042b0b3f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:16.684553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79465655-aae5-45fd-add1-bf26d759506a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:16.685239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79465655-aae5-45fd-add1-bf26d759506a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:16.685527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79465655-aae5-45fd-add1-bf26d759506a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:16.685819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79465655-aae5-45fd-add1-bf26d759506a None admin] PATH_INFO: `/v3/users/97ddd54bed064a388e7af44daf453763` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.687301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79465655-aae5-45fd-add1-bf26d759506a None admin] RBAC: Authorizing `identity:get_user(user_id=97ddd54bed064a388e7af44daf453763)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:16.689612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79465655-aae5-45fd-add1-bf26d759506a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:16.692225 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 184/358] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:05:16 2022] GET /identity/v3/users/97ddd54bed064a388e7af44daf453763 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:16.693084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79465655-aae5-45fd-add1-bf26d759506a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:16.777282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301238e-6105-4414-b6f5-f5788ce5408f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:16.825869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601a5a6-99ba-4025-b825-742c876f3c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:16.826612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:16.827135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601a5a6-99ba-4025-b825-742c876f3c9a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:16.827682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601a5a6-99ba-4025-b825-742c876f3c9a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:16.828143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601a5a6-99ba-4025-b825-742c876f3c9a None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.829001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601a5a6-99ba-4025-b825-742c876f3c9a None admin] RBAC: Adding query filter params (name=member) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:16.829494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601a5a6-99ba-4025-b825-742c876f3c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:16.831089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601a5a6-99ba-4025-b825-742c876f3c9a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:16.840735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79465655-aae5-45fd-add1-bf26d759506a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:16.850366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 175/359] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:16.850654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec42360-3ee3-49a1-99f4-e89f7132cda7 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:16.850784 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec42360-3ee3-49a1-99f4-e89f7132cda7 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:16.851029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec42360-3ee3-49a1-99f4-e89f7132cda7 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.857017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 176/360] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:16.857017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec42360-3ee3-49a1-99f4-e89f7132cda7 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:16.891704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986f176-2f5e-415b-866d-6192249d0be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:16.892700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986f176-2f5e-415b-866d-6192249d0be5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:16.893171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986f176-2f5e-415b-866d-6192249d0be5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:16.894472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986f176-2f5e-415b-866d-6192249d0be5 None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:16.896478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986f176-2f5e-415b-866d-6192249d0be5 None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:16.897931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986f176-2f5e-415b-866d-6192249d0be5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:16.899757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 185/361] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:16 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:16.903067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986f176-2f5e-415b-866d-6192249d0be5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:16.959273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:16.983208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec42360-3ee3-49a1-99f4-e89f7132cda7 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:17.025189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e64516-ce7a-4bb0-8070-928bbcf74fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:17.025797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e64516-ce7a-4bb0-8070-928bbcf74fe8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:17.026057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e64516-ce7a-4bb0-8070-928bbcf74fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:17.026329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e64516-ce7a-4bb0-8070-928bbcf74fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:17.027241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e64516-ce7a-4bb0-8070-928bbcf74fe8 None admin] RBAC: Adding query filter params (role.id=ef17b1e922a942a083c0f1905c8c523c, user.id=97ddd54bed064a388e7af44daf453763, scope.system=all) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:17.027537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e64516-ce7a-4bb0-8070-928bbcf74fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:17.029118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e64516-ce7a-4bb0-8070-928bbcf74fe8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:17.039752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986f176-2f5e-415b-866d-6192249d0be5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:17.046041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 177/362] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 22 17:05:16 2022] GET /identity/v3/role_assignments?user.id=97ddd54bed064a388e7af44daf453763&scope.system=all&role.id=ef17b1e922a942a083c0f1905c8c523c => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:17.046942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e64516-ce7a-4bb0-8070-928bbcf74fe8 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:17.096569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72cca2-6321-425e-9b4b-4c3a627e2dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:17.098054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72cca2-6321-425e-9b4b-4c3a627e2dd8 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:17.098569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72cca2-6321-425e-9b4b-4c3a627e2dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:17.099378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72cca2-6321-425e-9b4b-4c3a627e2dd8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:17.105028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:17.121036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72cca2-6321-425e-9b4b-4c3a627e2dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-9e72cca2-6321-425e-9b4b-4c3a627e2dd8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:17.133623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:17.134697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:17.135513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 186/363] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:05:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:17.143568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45eaf-ce72-4425-801d-facde688200e None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:17.144804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45eaf-ce72-4425-801d-facde688200e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:17.145277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45eaf-ce72-4425-801d-facde688200e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:17.169974 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-41d45eaf-ce72-4425-801d-facde688200e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:17.171365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-41d45eaf-ce72-4425-801d-facde688200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:17.217924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e64516-ce7a-4bb0-8070-928bbcf74fe8 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:17.263942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf4332-5cc2-4606-8bfa-85dbd8690b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:17.264553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf4332-5cc2-4606-8bfa-85dbd8690b30 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:17.264826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf4332-5cc2-4606-8bfa-85dbd8690b30 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:17.265095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf4332-5cc2-4606-8bfa-85dbd8690b30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:17.278095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 187/364] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:17.283658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45eaf-ce72-4425-801d-facde688200e None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:17.284717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:17.373516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf4332-5cc2-4606-8bfa-85dbd8690b30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:17.375576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf4332-5cc2-4606-8bfa-85dbd8690b30 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:17.435089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45eaf-ce72-4425-801d-facde688200e None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:17.480894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:17.491159 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4cd91-f118-49cc-8fb3-6c5269258b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:17.491348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4cd91-f118-49cc-8fb3-6c5269258b60 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:17.491571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4cd91-f118-49cc-8fb3-6c5269258b60 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:17.493597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4cd91-f118-49cc-8fb3-6c5269258b60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:17.493597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4cd91-f118-49cc-8fb3-6c5269258b60 None admin] RBAC: Adding query filter params (group.id=3348dd190d744e649fe289aaee2c6877, role.id=c2fe89b1b9fa47b19e02efd64ace9000, scope.project.id=c84516b7267f4b43bf816381e3e60ae6) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:17.493597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4cd91-f118-49cc-8fb3-6c5269258b60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:17.494274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4cd91-f118-49cc-8fb3-6c5269258b60 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:17.501390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 188/365] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 22 17:05:17 2022] GET /identity/v3/role_assignments?group.id=3348dd190d744e649fe289aaee2c6877&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=c2fe89b1b9fa47b19e02efd64ace9000 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:17.502335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4cd91-f118-49cc-8fb3-6c5269258b60 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:17.527559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 178/366] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:17.536280 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf4332-5cc2-4606-8bfa-85dbd8690b30 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:17.594327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:17.639239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4cd91-f118-49cc-8fb3-6c5269258b60 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:17.691233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf4332-5cc2-4606-8bfa-85dbd8690b30 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:17.704392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06893b-ae13-4e20-b63e-d55c9c2c2a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:17.705030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06893b-ae13-4e20-b63e-d55c9c2c2a0e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:17.705314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06893b-ae13-4e20-b63e-d55c9c2c2a0e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:17.705586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06893b-ae13-4e20-b63e-d55c9c2c2a0e None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:17.707411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06893b-ae13-4e20-b63e-d55c9c2c2a0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:17.707411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06893b-ae13-4e20-b63e-d55c9c2c2a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:17.708850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06893b-ae13-4e20-b63e-d55c9c2c2a0e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:17.726921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 189/367] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:17.727940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06893b-ae13-4e20-b63e-d55c9c2c2a0e None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:17.746977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b96d1-c09e-4d77-a54e-392009651a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:17.748513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b96d1-c09e-4d77-a54e-392009651a87 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:17.749143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b96d1-c09e-4d77-a54e-392009651a87 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:17.749603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b96d1-c09e-4d77-a54e-392009651a87 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:17.750816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b96d1-c09e-4d77-a54e-392009651a87 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:17.752330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b96d1-c09e-4d77-a54e-392009651a87 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:17.754141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 179/368] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:17 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:17.758536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b96d1-c09e-4d77-a54e-392009651a87 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:17.797875 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:17.860593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06893b-ae13-4e20-b63e-d55c9c2c2a0e None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:17.897769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b96d1-c09e-4d77-a54e-392009651a87 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:17.903309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641bb96-c3a7-43fb-9799-00cb2a1873b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:17.904687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641bb96-c3a7-43fb-9799-00cb2a1873b5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:17.905427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641bb96-c3a7-43fb-9799-00cb2a1873b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:17.906178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641bb96-c3a7-43fb-9799-00cb2a1873b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:17.908741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:17.920799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641bb96-c3a7-43fb-9799-00cb2a1873b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-a641bb96-c3a7-43fb-9799-00cb2a1873b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:17.927067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:17.928023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:05:17.928853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:17.931293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 190/369] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:05:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:17.932361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641bb96-c3a7-43fb-9799-00cb2a1873b5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:17.938717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130e057-4618-4c81-bcdf-cf5f71bc4b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:17.939339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130e057-4618-4c81-bcdf-cf5f71bc4b5e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:17.939618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130e057-4618-4c81-bcdf-cf5f71bc4b5e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:17.939899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130e057-4618-4c81-bcdf-cf5f71bc4b5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:18.043789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130e057-4618-4c81-bcdf-cf5f71bc4b5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:18.045630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130e057-4618-4c81-bcdf-cf5f71bc4b5e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:18.064532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641bb96-c3a7-43fb-9799-00cb2a1873b5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:18.082838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:18.109037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3a3e9-f401-4948-8583-5dbfaebd153d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:18.109921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3a3e9-f401-4948-8583-5dbfaebd153d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:18.110285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3a3e9-f401-4948-8583-5dbfaebd153d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:18.110686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3a3e9-f401-4948-8583-5dbfaebd153d None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:18.111946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3a3e9-f401-4948-8583-5dbfaebd153d None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:18.113760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3a3e9-f401-4948-8583-5dbfaebd153d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:18.115643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 191/370] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:17 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:18.119736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3a3e9-f401-4948-8583-5dbfaebd153d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:18.184317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 180/371] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:18.185823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130e057-4618-4c81-bcdf-cf5f71bc4b5e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:18.199002 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:18.255510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3a3e9-f401-4948-8583-5dbfaebd153d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:18.303796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f438a3-811a-464a-9b2d-8706188eaba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:18.304533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f438a3-811a-464a-9b2d-8706188eaba5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:18.304857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f438a3-811a-464a-9b2d-8706188eaba5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:18.305231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f438a3-811a-464a-9b2d-8706188eaba5 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:18.306077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f438a3-811a-464a-9b2d-8706188eaba5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:18.306498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f438a3-811a-464a-9b2d-8706188eaba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:18.308512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f438a3-811a-464a-9b2d-8706188eaba5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:18.320918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 192/372] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:18.323646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f438a3-811a-464a-9b2d-8706188eaba5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:18.338010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130e057-4618-4c81-bcdf-cf5f71bc4b5e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:18.391852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e671f0d-de28-49d0-ba6e-8fef6acc47ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:18.392767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e671f0d-de28-49d0-ba6e-8fef6acc47ce None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:18.393046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e671f0d-de28-49d0-ba6e-8fef6acc47ce None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:18.393392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e671f0d-de28-49d0-ba6e-8fef6acc47ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:18.396070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e671f0d-de28-49d0-ba6e-8fef6acc47ce None admin] RBAC: Adding query filter params (group.id=3348dd190d744e649fe289aaee2c6877, role.id=13accf89c12040508974db47c6905b31, scope.project.id=0a33f37b0d0d4ced9878417f46335942) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:18.396070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e671f0d-de28-49d0-ba6e-8fef6acc47ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:18.398731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e671f0d-de28-49d0-ba6e-8fef6acc47ce None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:18.401577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:18.406648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 181/373] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 22 17:05:18 2022] GET /identity/v3/role_assignments?group.id=3348dd190d744e649fe289aaee2c6877&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=13accf89c12040508974db47c6905b31 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:18.406648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e671f0d-de28-49d0-ba6e-8fef6acc47ce None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:18.483978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f438a3-811a-464a-9b2d-8706188eaba5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:18.514852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:18.537862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c3d14-30e6-470b-a900-1f934b57b819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:18.538942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c3d14-30e6-470b-a900-1f934b57b819 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:18.539439 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c3d14-30e6-470b-a900-1f934b57b819 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:18.539905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c3d14-30e6-470b-a900-1f934b57b819 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:18.541382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c3d14-30e6-470b-a900-1f934b57b819 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:18.543019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c3d14-30e6-470b-a900-1f934b57b819 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:18.547722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 193/374] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:18 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:18.547722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243f18f-f2d6-41dc-bd03-c0906b97568c None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:18.551017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243f18f-f2d6-41dc-bd03-c0906b97568c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:18.551437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243f18f-f2d6-41dc-bd03-c0906b97568c None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:18.554020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 194/375] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:18.555984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243f18f-f2d6-41dc-bd03-c0906b97568c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:18.593171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e671f0d-de28-49d0-ba6e-8fef6acc47ce None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:18.675635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a550c8-1a1b-4b64-b9d4-889fd454cad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:18.677483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a550c8-1a1b-4b64-b9d4-889fd454cad7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:18.678261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a550c8-1a1b-4b64-b9d4-889fd454cad7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:18.679038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a550c8-1a1b-4b64-b9d4-889fd454cad7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:18.687991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243f18f-f2d6-41dc-bd03-c0906b97568c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:18.700834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:18.731485 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453aec7-a582-4f1f-b5c6-66200c86e737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:18.732266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453aec7-a582-4f1f-b5c6-66200c86e737 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:18.732604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453aec7-a582-4f1f-b5c6-66200c86e737 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:18.732938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453aec7-a582-4f1f-b5c6-66200c86e737 None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:18.734660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453aec7-a582-4f1f-b5c6-66200c86e737 None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:18.736494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453aec7-a582-4f1f-b5c6-66200c86e737 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:18.742717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 195/376] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:18 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:18.746934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d505-e8df-468d-9e98-77cae1ae78a9 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:18.747670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d505-e8df-468d-9e98-77cae1ae78a9 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:18.748404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d505-e8df-468d-9e98-77cae1ae78a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:18.782233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6c59d505-e8df-468d-9e98-77cae1ae78a9 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:18.783529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6c59d505-e8df-468d-9e98-77cae1ae78a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:18.792628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a550c8-1a1b-4b64-b9d4-889fd454cad7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:18.794120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a550c8-1a1b-4b64-b9d4-889fd454cad7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:18.883736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 196/377] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:18.885313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:18.890811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59d505-e8df-468d-9e98-77cae1ae78a9 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:18.955755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 182/378] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:18.966689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a550c8-1a1b-4b64-b9d4-889fd454cad7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:19.045572 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59d505-e8df-468d-9e98-77cae1ae78a9 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:19.057845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:19.089030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef33a81-277a-4e53-b913-cef87587e8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:19.089799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef33a81-277a-4e53-b913-cef87587e8d7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:19.090071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef33a81-277a-4e53-b913-cef87587e8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:19.090348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef33a81-277a-4e53-b913-cef87587e8d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:19.091128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef33a81-277a-4e53-b913-cef87587e8d7 None admin] RBAC: Adding query filter params (group.id=3348dd190d744e649fe289aaee2c6877, role.id=c2fe89b1b9fa47b19e02efd64ace9000, scope.project.id=0a33f37b0d0d4ced9878417f46335942) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:19.091440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef33a81-277a-4e53-b913-cef87587e8d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:19.093117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef33a81-277a-4e53-b913-cef87587e8d7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:19.096425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a550c8-1a1b-4b64-b9d4-889fd454cad7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:19.101086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 197/379] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 22 17:05:18 2022] GET /identity/v3/role_assignments?group.id=3348dd190d744e649fe289aaee2c6877&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=c2fe89b1b9fa47b19e02efd64ace9000 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:19.140226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436428c4-2c79-47da-a138-215749a36a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:19.140826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436428c4-2c79-47da-a138-215749a36a2f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:19.141127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436428c4-2c79-47da-a138-215749a36a2f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:19.141404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436428c4-2c79-47da-a138-215749a36a2f None admin] PATH_INFO: `/v3/projects/14e122ee73be44deb3dfe27923205d92` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:19.142786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436428c4-2c79-47da-a138-215749a36a2f None admin] RBAC: Authorizing `identity:get_project(project_id=14e122ee73be44deb3dfe27923205d92)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:19.145324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436428c4-2c79-47da-a138-215749a36a2f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:19.147608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 183/380] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:18 2022] GET /identity/v3/projects/14e122ee73be44deb3dfe27923205d92 => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:19.157041 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef33a81-277a-4e53-b913-cef87587e8d7 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:19.306282 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef33a81-277a-4e53-b913-cef87587e8d7 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:19.353150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fb84b-cb24-4642-9059-a1959fd6197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:19.354167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb84b-cb24-4642-9059-a1959fd6197b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:19.354688 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb84b-cb24-4642-9059-a1959fd6197b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:19.355151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb84b-cb24-4642-9059-a1959fd6197b None admin] PATH_INFO: `/v3/groups/cba2b2fe80da419d9e2134d21b087500` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:19.364434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fb84b-cb24-4642-9059-a1959fd6197b None admin] RBAC: Authorizing `identity:get_group(group_id=cba2b2fe80da419d9e2134d21b087500)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:19.366337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fb84b-cb24-4642-9059-a1959fd6197b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:19.368630 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 198/381] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:19 2022] GET /identity/v3/groups/cba2b2fe80da419d9e2134d21b087500 => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:19.375113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436428c4-2c79-47da-a138-215749a36a2f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:19.378769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fb84b-cb24-4642-9059-a1959fd6197b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:19.416751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:19.458111 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:19.555546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fb84b-cb24-4642-9059-a1959fd6197b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:19.595182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436428c4-2c79-47da-a138-215749a36a2f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:19.635018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a7243-7fac-4ac4-b924-a1763b334198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:19.635670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a7243-7fac-4ac4-b924-a1763b334198 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:19.635945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a7243-7fac-4ac4-b924-a1763b334198 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:19.636213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a7243-7fac-4ac4-b924-a1763b334198 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:19.663933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a7243-7fac-4ac4-b924-a1763b334198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-6e5a7243-7fac-4ac4-b924-a1763b334198 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:19.672596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:19.673533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:19.674337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:19.676005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ddd73-5f04-4f08-b1ec-a54fa21e69d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:19.676005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ddd73-5f04-4f08-b1ec-a54fa21e69d3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:19.676005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ddd73-5f04-4f08-b1ec-a54fa21e69d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:19.676392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ddd73-5f04-4f08-b1ec-a54fa21e69d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:19.682518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ddd73-5f04-4f08-b1ec-a54fa21e69d3 None admin] RBAC: Adding query filter params (group.id=cba2b2fe80da419d9e2134d21b087500, role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=14e122ee73be44deb3dfe27923205d92) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:19.682518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ddd73-5f04-4f08-b1ec-a54fa21e69d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:19.682883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 199/382] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:19.683017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ddd73-5f04-4f08-b1ec-a54fa21e69d3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:19.684027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a7243-7fac-4ac4-b924-a1763b334198 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:19.692360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 184/383] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 22 17:05:19 2022] GET /identity/v3/role_assignments?group.id=cba2b2fe80da419d9e2134d21b087500&scope.project.id=14e122ee73be44deb3dfe27923205d92&role.id=4b10782d99b74f5bb453b82286012e0c => generated 260 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:19.801613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:19.808626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a7243-7fac-4ac4-b924-a1763b334198 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:19.851958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2005ac2-a3b5-4847-a0eb-6eff40964880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:19.852590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2005ac2-a3b5-4847-a0eb-6eff40964880 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:19.852869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2005ac2-a3b5-4847-a0eb-6eff40964880 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:19.853138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2005ac2-a3b5-4847-a0eb-6eff40964880 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:19.853819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2005ac2-a3b5-4847-a0eb-6eff40964880 None admin] RBAC: Adding query filter params (name=member) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:19.854120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2005ac2-a3b5-4847-a0eb-6eff40964880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:19.855548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2005ac2-a3b5-4847-a0eb-6eff40964880 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:19.865538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 200/384] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:19.871490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ddd73-5f04-4f08-b1ec-a54fa21e69d3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:20.015866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ddd73-5f04-4f08-b1ec-a54fa21e69d3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:20.018267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:20.057714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31fa4c-9dcc-4954-8130-0e76d1ce2012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:20.058349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31fa4c-9dcc-4954-8130-0e76d1ce2012 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:20.058727 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31fa4c-9dcc-4954-8130-0e76d1ce2012 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:20.059000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31fa4c-9dcc-4954-8130-0e76d1ce2012 None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:20.062950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31fa4c-9dcc-4954-8130-0e76d1ce2012 None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:20.064562 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31fa4c-9dcc-4954-8130-0e76d1ce2012 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:20.067315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 185/385] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:19 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:20.072979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2005ac2-a3b5-4847-a0eb-6eff40964880 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:20.212141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2005ac2-a3b5-4847-a0eb-6eff40964880 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:20.252564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fcb15-b9d7-47aa-8cf3-54c609cece99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:20.253257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fcb15-b9d7-47aa-8cf3-54c609cece99 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:20.253533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fcb15-b9d7-47aa-8cf3-54c609cece99 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:20.253800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fcb15-b9d7-47aa-8cf3-54c609cece99 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:20.255191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fcb15-b9d7-47aa-8cf3-54c609cece99 None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:20.256626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fcb15-b9d7-47aa-8cf3-54c609cece99 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:20.258632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 201/386] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:20 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:20.264473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31fa4c-9dcc-4954-8130-0e76d1ce2012 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:20.312549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80ccdd-8575-4997-9c4b-d76030678eb0 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:20.312903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80ccdd-8575-4997-9c4b-d76030678eb0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:20.313078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80ccdd-8575-4997-9c4b-d76030678eb0 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:20.315262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 202/387] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:20.324829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecbb84-283a-47ba-b5bc-a6c8f0666dce None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:20.325122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecbb84-283a-47ba-b5bc-a6c8f0666dce None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:20.325401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecbb84-283a-47ba-b5bc-a6c8f0666dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:20.364757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6fecbb84-283a-47ba-b5bc-a6c8f0666dce None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:20.366015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6fecbb84-283a-47ba-b5bc-a6c8f0666dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:20.404928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:20.473889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 203/388] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:20.489198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31fa4c-9dcc-4954-8130-0e76d1ce2012 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:20.534260 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83959ad2-e738-4fb0-94e4-ca0bb270297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:20.535188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83959ad2-e738-4fb0-94e4-ca0bb270297d None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:20.535463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83959ad2-e738-4fb0-94e4-ca0bb270297d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:20.535731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83959ad2-e738-4fb0-94e4-ca0bb270297d None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6/groups/3348dd190d744e649fe289aaee2c6877/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:20.539907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83959ad2-e738-4fb0-94e4-ca0bb270297d None admin] RBAC: Authorizing `identity:create_grant(project_id=c84516b7267f4b43bf816381e3e60ae6, group_id=3348dd190d744e649fe289aaee2c6877, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:20.541430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83959ad2-e738-4fb0-94e4-ca0bb270297d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:20.637034 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 186/389] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 22 17:05:20 2022] PUT /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6/groups/3348dd190d744e649fe289aaee2c6877/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:05:20.790667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecbb84-283a-47ba-b5bc-a6c8f0666dce None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:20.934057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecbb84-283a-47ba-b5bc-a6c8f0666dce None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:20.978213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bd05b-e06c-468d-a598-2094ae2c93a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:20.978902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bd05b-e06c-468d-a598-2094ae2c93a5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:20.979178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bd05b-e06c-468d-a598-2094ae2c93a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:20.979451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bd05b-e06c-468d-a598-2094ae2c93a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:20.991772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bd05b-e06c-468d-a598-2094ae2c93a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-590bd05b-e06c-468d-a598-2094ae2c93a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:20.996574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:20.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:20.998288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 204/390] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:05:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:21.004329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83959ad2-e738-4fb0-94e4-ca0bb270297d None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:21.063934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:21.157782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83959ad2-e738-4fb0-94e4-ca0bb270297d None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:21.204824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49497a-5cc7-42db-9ae1-28a173ea5e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:21.205366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49497a-5cc7-42db-9ae1-28a173ea5e3b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:21.205641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49497a-5cc7-42db-9ae1-28a173ea5e3b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:21.206014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49497a-5cc7-42db-9ae1-28a173ea5e3b None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:21.207460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49497a-5cc7-42db-9ae1-28a173ea5e3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:21.207764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49497a-5cc7-42db-9ae1-28a173ea5e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:21.209179 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49497a-5cc7-42db-9ae1-28a173ea5e3b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:21.222715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 187/391] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:21.228573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bd05b-e06c-468d-a598-2094ae2c93a5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:21.253505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144465fb-56c0-424f-a964-e6931c78064f None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:21.253915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144465fb-56c0-424f-a964-e6931c78064f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:21.254079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144465fb-56c0-424f-a964-e6931c78064f None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:21.257355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 188/392] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:21.265228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb9ec1-d7e8-44dc-a8cd-64b99e6808bb None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:21.266074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb9ec1-d7e8-44dc-a8cd-64b99e6808bb None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:21.266074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb9ec1-d7e8-44dc-a8cd-64b99e6808bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:21.271730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:21.294686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-7ebb9ec1-d7e8-44dc-a8cd-64b99e6808bb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:21.295919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7ebb9ec1-d7e8-44dc-a8cd-64b99e6808bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:21.420898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 189/393] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:21.431169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bd05b-e06c-468d-a598-2094ae2c93a5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:21.454470 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883318c4-16ad-452d-960f-2ccdf602df7e None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:21.454807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883318c4-16ad-452d-960f-2ccdf602df7e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:21.455262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883318c4-16ad-452d-960f-2ccdf602df7e None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:21.457717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 190/394] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:05:21.463902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12ea17-d43b-4495-a5cd-c0819c7705c2 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:21.464671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12ea17-d43b-4495-a5cd-c0819c7705c2 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:21.465180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12ea17-d43b-4495-a5cd-c0819c7705c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:21.479335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb66899-69a8-469a-9ce8-be1e401d1b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:21.479973 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb66899-69a8-469a-9ce8-be1e401d1b47 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:21.480246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb66899-69a8-469a-9ce8-be1e401d1b47 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:21.480515 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb66899-69a8-469a-9ce8-be1e401d1b47 None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:21.481669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb66899-69a8-469a-9ce8-be1e401d1b47 None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:21.483769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb66899-69a8-469a-9ce8-be1e401d1b47 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:21.486111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 205/395] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:21 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:21.493126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-6e12ea17-d43b-4495-a5cd-c0819c7705c2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:21.493126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-6e12ea17-d43b-4495-a5cd-c0819c7705c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:21.495488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb66899-69a8-469a-9ce8-be1e401d1b47 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:21.515477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:21.575948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 191/396] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:21.592143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:21.610182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb66899-69a8-469a-9ce8-be1e401d1b47 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:21.648621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de320552-ddb7-48aa-955a-293d90854fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:21.649260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de320552-ddb7-48aa-955a-293d90854fac None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:21.649537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de320552-ddb7-48aa-955a-293d90854fac None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:21.649806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de320552-ddb7-48aa-955a-293d90854fac None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:21.651029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de320552-ddb7-48aa-955a-293d90854fac None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:21.652467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de320552-ddb7-48aa-955a-293d90854fac None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:21.654418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 206/397] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:21 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:21.660674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12ea17-d43b-4495-a5cd-c0819c7705c2 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:21.775178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12ea17-d43b-4495-a5cd-c0819c7705c2 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:21.829364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009bd49-7f5a-4454-b7aa-4cca630e0a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:21.830397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009bd49-7f5a-4454-b7aa-4cca630e0a16 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:21.830911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009bd49-7f5a-4454-b7aa-4cca630e0a16 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:21.831376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009bd49-7f5a-4454-b7aa-4cca630e0a16 None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6/groups/3348dd190d744e649fe289aaee2c6877/roles/c2fe89b1b9fa47b19e02efd64ace9000` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:21.833905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009bd49-7f5a-4454-b7aa-4cca630e0a16 None admin] RBAC: Authorizing `identity:create_grant(project_id=c84516b7267f4b43bf816381e3e60ae6, group_id=3348dd190d744e649fe289aaee2c6877, role_id=c2fe89b1b9fa47b19e02efd64ace9000)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:21.835634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009bd49-7f5a-4454-b7aa-4cca630e0a16 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:21.932674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de320552-ddb7-48aa-955a-293d90854fac None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:21.939621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 192/398] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 22 17:05:21 2022] PUT /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6/groups/3348dd190d744e649fe289aaee2c6877/roles/c2fe89b1b9fa47b19e02efd64ace9000 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:05:21.942261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009bd49-7f5a-4454-b7aa-4cca630e0a16 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:21.974407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:22.109658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de320552-ddb7-48aa-955a-293d90854fac None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:22.172227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009bd49-7f5a-4454-b7aa-4cca630e0a16 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:22.188862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f7ed0-017e-4fab-a91e-0c0fd88a32de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:22.189483 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7ed0-017e-4fab-a91e-0c0fd88a32de None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:22.189763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7ed0-017e-4fab-a91e-0c0fd88a32de None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:22.190030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7ed0-017e-4fab-a91e-0c0fd88a32de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:22.203730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f7ed0-017e-4fab-a91e-0c0fd88a32de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-938f7ed0-017e-4fab-a91e-0c0fd88a32de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:22.210459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:22.211456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:22.212260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:22.213358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 207/399] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:05:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:22.218289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f7ed0-017e-4fab-a91e-0c0fd88a32de None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:22.230904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81d201-9abe-4c00-bfc3-bedf90a7827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:22.231563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81d201-9abe-4c00-bfc3-bedf90a7827e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:22.231905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81d201-9abe-4c00-bfc3-bedf90a7827e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:22.232178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81d201-9abe-4c00-bfc3-bedf90a7827e None admin] PATH_INFO: `/v3/roles/member` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:22.241185 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:22.256912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81d201-9abe-4c00-bfc3-bedf90a7827e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-de81d201-9abe-4c00-bfc3-bedf90a7827e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:22.263591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:22.264577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:22.265947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 193/400] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:22.271517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81d201-9abe-4c00-bfc3-bedf90a7827e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:22.382886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:22.431859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81d201-9abe-4c00-bfc3-bedf90a7827e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:22.432774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f7ed0-017e-4fab-a91e-0c0fd88a32de None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:22.476689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b233dd-bcf7-40bf-a728-943e729507df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:22.477302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b233dd-bcf7-40bf-a728-943e729507df None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:22.477581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b233dd-bcf7-40bf-a728-943e729507df None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:22.477846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b233dd-bcf7-40bf-a728-943e729507df None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:22.478539 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b233dd-bcf7-40bf-a728-943e729507df None admin] RBAC: Adding query filter params (name=member) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:22.478843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b233dd-bcf7-40bf-a728-943e729507df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:22.480244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b233dd-bcf7-40bf-a728-943e729507df None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:22.491249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b4955-199e-44ac-b0e1-c0a2394001f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:22.491868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4955-199e-44ac-b0e1-c0a2394001f3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:22.492146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4955-199e-44ac-b0e1-c0a2394001f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:22.492476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4955-199e-44ac-b0e1-c0a2394001f3 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:22.493154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b4955-199e-44ac-b0e1-c0a2394001f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:22.493480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b4955-199e-44ac-b0e1-c0a2394001f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:22.494909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b4955-199e-44ac-b0e1-c0a2394001f3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:22.497255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 194/401] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:22.498602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d79ccb-6220-4afc-af20-63a4e77813b9 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:22.498887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d79ccb-6220-4afc-af20-63a4e77813b9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:22.499156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d79ccb-6220-4afc-af20-63a4e77813b9 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:22.505125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 195/402] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:22.506054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d79ccb-6220-4afc-af20-63a4e77813b9 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:22.510859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 208/403] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:22.513311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9722d8-87c3-4471-87dc-ed954275ab0a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:22.513593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9722d8-87c3-4471-87dc-ed954275ab0a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:22.513868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9722d8-87c3-4471-87dc-ed954275ab0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:22.538210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1e9722d8-87c3-4471-87dc-ed954275ab0a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:22.539723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1e9722d8-87c3-4471-87dc-ed954275ab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:22.638637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 209/404] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:22.638637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9722d8-87c3-4471-87dc-ed954275ab0a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:22.638637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:22.706397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d79ccb-6220-4afc-af20-63a4e77813b9 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:22.760275 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9722d8-87c3-4471-87dc-ed954275ab0a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:22.780138 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f2dc2-33ed-4d86-9c68-d2a6981f9c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:22.781073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f2dc2-33ed-4d86-9c68-d2a6981f9c2f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:22.781501 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f2dc2-33ed-4d86-9c68-d2a6981f9c2f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:22.781916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f2dc2-33ed-4d86-9c68-d2a6981f9c2f None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:22.787378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f2dc2-33ed-4d86-9c68-d2a6981f9c2f None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:22.789121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f2dc2-33ed-4d86-9c68-d2a6981f9c2f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:22.791048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 196/405] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:22 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:22.797246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f2dc2-33ed-4d86-9c68-d2a6981f9c2f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:22.806675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7530eb-1b00-4a76-8ac6-4dd081c49ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:22.807701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7530eb-1b00-4a76-8ac6-4dd081c49ed8 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:22.809143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:22.810266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7530eb-1b00-4a76-8ac6-4dd081c49ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:22.810720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7530eb-1b00-4a76-8ac6-4dd081c49ed8 None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:22.812107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7530eb-1b00-4a76-8ac6-4dd081c49ed8 None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:22.813980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7530eb-1b00-4a76-8ac6-4dd081c49ed8 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:22.816753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 210/406] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:22 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:22.823642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7530eb-1b00-4a76-8ac6-4dd081c49ed8 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:22.907584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:22.945926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7530eb-1b00-4a76-8ac6-4dd081c49ed8 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:22.993408 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf69bc-5fb1-4ca9-a9da-bba924b30920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:22.994287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf69bc-5fb1-4ca9-a9da-bba924b30920 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:22.994809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf69bc-5fb1-4ca9-a9da-bba924b30920 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:22.995236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf69bc-5fb1-4ca9-a9da-bba924b30920 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:22.997675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf69bc-5fb1-4ca9-a9da-bba924b30920 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:22.999518 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf69bc-5fb1-4ca9-a9da-bba924b30920 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:23.001426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 211/407] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:22 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:23.009319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf69bc-5fb1-4ca9-a9da-bba924b30920 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:23.054736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f2dc2-33ed-4d86-9c68-d2a6981f9c2f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:23.098246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd8ec8-a951-4ac8-9353-9fdc014ca3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:23.099035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd8ec8-a951-4ac8-9353-9fdc014ca3d8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:23.099325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd8ec8-a951-4ac8-9353-9fdc014ca3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:23.099630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd8ec8-a951-4ac8-9353-9fdc014ca3d8 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:23.101945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd8ec8-a951-4ac8-9353-9fdc014ca3d8 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:23.103511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd8ec8-a951-4ac8-9353-9fdc014ca3d8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:23.106392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 197/408] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:22 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:23.111780 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd8ec8-a951-4ac8-9353-9fdc014ca3d8 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:23.145928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf69bc-5fb1-4ca9-a9da-bba924b30920 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:23.211690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:23.217520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1e62e-625f-4da6-bbad-f56a56fefad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:23.218648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1e62e-625f-4da6-bbad-f56a56fefad9 None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:23.219009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1e62e-625f-4da6-bbad-f56a56fefad9 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:23.219489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1e62e-625f-4da6-bbad-f56a56fefad9 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942/groups/3348dd190d744e649fe289aaee2c6877/roles/c2fe89b1b9fa47b19e02efd64ace9000` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:23.226059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1e62e-625f-4da6-bbad-f56a56fefad9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a33f37b0d0d4ced9878417f46335942, group_id=3348dd190d744e649fe289aaee2c6877, role_id=c2fe89b1b9fa47b19e02efd64ace9000)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:23.230526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1e62e-625f-4da6-bbad-f56a56fefad9 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:23.251281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd8ec8-a951-4ac8-9353-9fdc014ca3d8 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:23.289510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59409ff-3c46-46ea-a33f-8327cb76062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:23.290207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59409ff-3c46-46ea-a33f-8327cb76062b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:23.290510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59409ff-3c46-46ea-a33f-8327cb76062b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:23.290714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59409ff-3c46-46ea-a33f-8327cb76062b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:23.303545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59409ff-3c46-46ea-a33f-8327cb76062b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-e59409ff-3c46-46ea-a33f-8327cb76062b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:23.308850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:23.309786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:23.310627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 198/409] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:05:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:23.311340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59409ff-3c46-46ea-a33f-8327cb76062b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:23.355995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:23.360936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 212/410] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 22 17:05:23 2022] PUT /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942/groups/3348dd190d744e649fe289aaee2c6877/roles/c2fe89b1b9fa47b19e02efd64ace9000 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:05:23.362775 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1e62e-625f-4da6-bbad-f56a56fefad9 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:23.483415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59409ff-3c46-46ea-a33f-8327cb76062b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:23.520330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1e62e-625f-4da6-bbad-f56a56fefad9 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:23.525144 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:23.530381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d782f8-62a6-4c7a-824e-afa2a9d9eef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:23.541112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d782f8-62a6-4c7a-824e-afa2a9d9eef5 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:23.541112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d782f8-62a6-4c7a-824e-afa2a9d9eef5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:23.541112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d782f8-62a6-4c7a-824e-afa2a9d9eef5 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942/groups/3348dd190d744e649fe289aaee2c6877/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:23.541112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d782f8-62a6-4c7a-824e-afa2a9d9eef5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a33f37b0d0d4ced9878417f46335942, group_id=3348dd190d744e649fe289aaee2c6877, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:23.541112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d782f8-62a6-4c7a-824e-afa2a9d9eef5 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:23.572806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d4ba2-366e-4aa2-99c8-8eb58c65ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:23.573505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d4ba2-366e-4aa2-99c8-8eb58c65ba5f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:23.573776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d4ba2-366e-4aa2-99c8-8eb58c65ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:23.574053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d4ba2-366e-4aa2-99c8-8eb58c65ba5f None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:23.574749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d4ba2-366e-4aa2-99c8-8eb58c65ba5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:23.577174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d4ba2-366e-4aa2-99c8-8eb58c65ba5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:23.577174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d4ba2-366e-4aa2-99c8-8eb58c65ba5f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:23.591252 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 213/411] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:23.592706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5e290-61d4-4240-a245-cec82ccc7ad1 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:23.592996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5e290-61d4-4240-a245-cec82ccc7ad1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:23.593274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5e290-61d4-4240-a245-cec82ccc7ad1 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:23.595574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 214/412] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:23.601139 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5e290-61d4-4240-a245-cec82ccc7ad1 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:23.680752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 199/413] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 22 17:05:23 2022] PUT /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942/groups/3348dd190d744e649fe289aaee2c6877/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:05:23.684668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5201a64-9f71-4a24-a06b-223a1dcca1f9 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:23.685986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5201a64-9f71-4a24-a06b-223a1dcca1f9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:23.687270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:23.688963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5201a64-9f71-4a24-a06b-223a1dcca1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:23.727800 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a5201a64-9f71-4a24-a06b-223a1dcca1f9 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:23.730603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a5201a64-9f71-4a24-a06b-223a1dcca1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:23.759234 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5e290-61d4-4240-a245-cec82ccc7ad1 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:23.839195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296295e-db65-4744-9a97-2febeca3a213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:23.840681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296295e-db65-4744-9a97-2febeca3a213 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:23.841436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296295e-db65-4744-9a97-2febeca3a213 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:23.842159 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296295e-db65-4744-9a97-2febeca3a213 None admin] PATH_INFO: `/v3/groups/cba2b2fe80da419d9e2134d21b087500` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:23.844590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296295e-db65-4744-9a97-2febeca3a213 None admin] RBAC: Authorizing `identity:get_group(group_id=cba2b2fe80da419d9e2134d21b087500)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:23.848111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296295e-db65-4744-9a97-2febeca3a213 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:23.852698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 215/414] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:23 2022] GET /identity/v3/groups/cba2b2fe80da419d9e2134d21b087500 => generated 229 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:23.861614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296295e-db65-4744-9a97-2febeca3a213 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:23.900220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 200/415] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:24.060404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296295e-db65-4744-9a97-2febeca3a213 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:24.092827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:24.101213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f24570-b2ea-4083-8181-f641d39916e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:24.102178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f24570-b2ea-4083-8181-f641d39916e7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:24.102547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f24570-b2ea-4083-8181-f641d39916e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:24.108663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f24570-b2ea-4083-8181-f641d39916e7 None admin] PATH_INFO: `/v3/projects/14e122ee73be44deb3dfe27923205d92` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:24.108663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f24570-b2ea-4083-8181-f641d39916e7 None admin] RBAC: Authorizing `identity:get_project(project_id=14e122ee73be44deb3dfe27923205d92)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:24.108663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f24570-b2ea-4083-8181-f641d39916e7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:24.108663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 216/416] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:23 2022] GET /identity/v3/projects/14e122ee73be44deb3dfe27923205d92 => generated 344 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:24.114451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5201a64-9f71-4a24-a06b-223a1dcca1f9 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:24.239526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5201a64-9f71-4a24-a06b-223a1dcca1f9 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:24.264243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:24.283357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e425e4d-412d-4872-808e-207de1cb568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:24.284631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e425e4d-412d-4872-808e-207de1cb568a None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:24.285084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e425e4d-412d-4872-808e-207de1cb568a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:24.285474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74b068-80d4-47e4-8593-e49972c4c4ff None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:24.285824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e425e4d-412d-4872-808e-207de1cb568a None admin] PATH_INFO: `/v3/projects/14e122ee73be44deb3dfe27923205d92/groups/cba2b2fe80da419d9e2134d21b087500/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:24.285918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74b068-80d4-47e4-8593-e49972c4c4ff None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:24.286072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74b068-80d4-47e4-8593-e49972c4c4ff None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:24.287669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 217/417] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:24.287785 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e425e4d-412d-4872-808e-207de1cb568a None admin] RBAC: Authorizing `identity:create_grant(project_id=14e122ee73be44deb3dfe27923205d92, group_id=cba2b2fe80da419d9e2134d21b087500, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:24.289305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e425e4d-412d-4872-808e-207de1cb568a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:24.293401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b9e86-3639-41ba-85b1-6a6ba02d9ca1 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:24.293696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b9e86-3639-41ba-85b1-6a6ba02d9ca1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:24.293994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b9e86-3639-41ba-85b1-6a6ba02d9ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:24.319535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-ff5b9e86-3639-41ba-85b1-6a6ba02d9ca1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:24.320655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ff5b9e86-3639-41ba-85b1-6a6ba02d9ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:24.384209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 201/418] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 22 17:05:24 2022] PUT /identity/v3/projects/14e122ee73be44deb3dfe27923205d92/groups/cba2b2fe80da419d9e2134d21b087500/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:05:24.404373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e425e4d-412d-4872-808e-207de1cb568a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:24.440467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 218/419] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:24.596208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e425e4d-412d-4872-808e-207de1cb568a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:24.634836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ed3d-7ae1-4d1b-aaad-f0f347563fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:24.635475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ed3d-7ae1-4d1b-aaad-f0f347563fbf None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:24.635747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ed3d-7ae1-4d1b-aaad-f0f347563fbf None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:24.636012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ed3d-7ae1-4d1b-aaad-f0f347563fbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:24.647914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708ed3d-7ae1-4d1b-aaad-f0f347563fbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-f708ed3d-7ae1-4d1b-aaad-f0f347563fbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:24.653388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:24.654362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:24.655224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 202/420] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:24.661171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b9e86-3639-41ba-85b1-6a6ba02d9ca1 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:24.798683 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b9e86-3639-41ba-85b1-6a6ba02d9ca1 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:24.844522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:24.857610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463c5e-2664-445b-914f-cdc1b1998e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:24.857610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463c5e-2664-445b-914f-cdc1b1998e31 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:24.857610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463c5e-2664-445b-914f-cdc1b1998e31 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:24.857610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463c5e-2664-445b-914f-cdc1b1998e31 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:24.858095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463c5e-2664-445b-914f-cdc1b1998e31 None admin] RBAC: Adding query filter params (name=member) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:24.858095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463c5e-2664-445b-914f-cdc1b1998e31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:24.858095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463c5e-2664-445b-914f-cdc1b1998e31 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:24.860706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ed3d-7ae1-4d1b-aaad-f0f347563fbf None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:24.871506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 219/421] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:24.872946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463c5e-2664-445b-914f-cdc1b1998e31 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:24.959342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:25.028156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ed3d-7ae1-4d1b-aaad-f0f347563fbf None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:25.050468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463c5e-2664-445b-914f-cdc1b1998e31 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:25.085556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e534de-191c-4d31-a73a-ef1891a214e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:25.086625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e534de-191c-4d31-a73a-ef1891a214e9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:25.087121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e534de-191c-4d31-a73a-ef1891a214e9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:25.087595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e534de-191c-4d31-a73a-ef1891a214e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:25.110731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e534de-191c-4d31-a73a-ef1891a214e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-26e534de-191c-4d31-a73a-ef1891a214e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:25.125432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:25.127254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:25.128544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 203/422] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:05:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:25.136394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e534de-191c-4d31-a73a-ef1891a214e9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:25.143636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7fae2-a6c4-443c-9f59-75aac5d5782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:25.144268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7fae2-a6c4-443c-9f59-75aac5d5782f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:25.144545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7fae2-a6c4-443c-9f59-75aac5d5782f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:25.144813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7fae2-a6c4-443c-9f59-75aac5d5782f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:25.270588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:25.282014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7fae2-a6c4-443c-9f59-75aac5d5782f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:25.283754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7fae2-a6c4-443c-9f59-75aac5d5782f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:25.364617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e534de-191c-4d31-a73a-ef1891a214e9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:25.370121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:25.437592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23597c23-6966-4a0b-8def-71549eba0b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:25.438954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23597c23-6966-4a0b-8def-71549eba0b05 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:25.439972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23597c23-6966-4a0b-8def-71549eba0b05 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:25.440999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23597c23-6966-4a0b-8def-71549eba0b05 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:25.442162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23597c23-6966-4a0b-8def-71549eba0b05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:25.442735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23597c23-6966-4a0b-8def-71549eba0b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:25.445536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23597c23-6966-4a0b-8def-71549eba0b05 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:25.461024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 220/423] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 17:05:25.461581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 204/424] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:25.472660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7fae2-a6c4-443c-9f59-75aac5d5782f None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:25.475656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23597c23-6966-4a0b-8def-71549eba0b05 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:25.680088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23597c23-6966-4a0b-8def-71549eba0b05 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:25.689745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:25.708770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7fae2-a6c4-443c-9f59-75aac5d5782f None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:25.722416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743b28e-07a7-4ed0-beda-73dc1666abda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:25.723063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743b28e-07a7-4ed0-beda-73dc1666abda None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:25.723338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743b28e-07a7-4ed0-beda-73dc1666abda None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:25.723613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743b28e-07a7-4ed0-beda-73dc1666abda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:25.788548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fc49f-5a29-4999-8efe-7fff217a145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:25.789191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fc49f-5a29-4999-8efe-7fff217a145e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:25.789471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fc49f-5a29-4999-8efe-7fff217a145e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:25.789746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fc49f-5a29-4999-8efe-7fff217a145e None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:25.792150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fc49f-5a29-4999-8efe-7fff217a145e None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:25.793650 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fc49f-5a29-4999-8efe-7fff217a145e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:25.794840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:25.795595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 221/425] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:25 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:25.802821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fc49f-5a29-4999-8efe-7fff217a145e None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:25.850905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743b28e-07a7-4ed0-beda-73dc1666abda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:25.852832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743b28e-07a7-4ed0-beda-73dc1666abda None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:25.943297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fc49f-5a29-4999-8efe-7fff217a145e None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:25.983023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68359c47-6ffc-4f13-9436-4234be2cb41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:25.983658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359c47-6ffc-4f13-9436-4234be2cb41e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:25.983930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359c47-6ffc-4f13-9436-4234be2cb41e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:25.984202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359c47-6ffc-4f13-9436-4234be2cb41e None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:25.985355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68359c47-6ffc-4f13-9436-4234be2cb41e None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:25.987548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68359c47-6ffc-4f13-9436-4234be2cb41e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:25.989904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 222/426] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:25 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:25.991128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223ef5e-cd1f-41d2-ae92-5c03ff032906 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:25.991402 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223ef5e-cd1f-41d2-ae92-5c03ff032906 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:25.991668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223ef5e-cd1f-41d2-ae92-5c03ff032906 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:25.992943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 223/427] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:25.997805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223ef5e-cd1f-41d2-ae92-5c03ff032906 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:26.011995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 205/428] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:26.013344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d188c-1ac3-4b41-a0d5-e21f7c62f785 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:26.013899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d188c-1ac3-4b41-a0d5-e21f7c62f785 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:26.014182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d188c-1ac3-4b41-a0d5-e21f7c62f785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:26.033285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:26.037293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-623d188c-1ac3-4b41-a0d5-e21f7c62f785 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:26.038518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-623d188c-1ac3-4b41-a0d5-e21f7c62f785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:26.107639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:26.116780 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 206/429] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:26.117635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d188c-1ac3-4b41-a0d5-e21f7c62f785 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:26.134437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223ef5e-cd1f-41d2-ae92-5c03ff032906 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:26.180919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99383e-ab55-4a79-9f02-6f8ed61a0128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:26.181772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99383e-ab55-4a79-9f02-6f8ed61a0128 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:26.182190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99383e-ab55-4a79-9f02-6f8ed61a0128 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:26.182790 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99383e-ab55-4a79-9f02-6f8ed61a0128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:26.183677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99383e-ab55-4a79-9f02-6f8ed61a0128 None admin] RBAC: Adding query filter params (group.id=3348dd190d744e649fe289aaee2c6877, role.id=13accf89c12040508974db47c6905b31, scope.project.id=c84516b7267f4b43bf816381e3e60ae6) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:26.184377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99383e-ab55-4a79-9f02-6f8ed61a0128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:26.185980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99383e-ab55-4a79-9f02-6f8ed61a0128 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:26.200271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 224/430] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 22 17:05:25 2022] GET /identity/v3/role_assignments?group.id=3348dd190d744e649fe289aaee2c6877&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=13accf89c12040508974db47c6905b31 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:26.321636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d188c-1ac3-4b41-a0d5-e21f7c62f785 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:26.368324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a17b5-b056-4916-b5fa-ad3dc8dd867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:26.369345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a17b5-b056-4916-b5fa-ad3dc8dd867d None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:26.369821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a17b5-b056-4916-b5fa-ad3dc8dd867d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:26.370311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a17b5-b056-4916-b5fa-ad3dc8dd867d None admin] PATH_INFO: `/v3/projects/c84516b7267f4b43bf816381e3e60ae6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:26.371716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a17b5-b056-4916-b5fa-ad3dc8dd867d None admin] RBAC: Authorizing `identity:get_project(project_id=c84516b7267f4b43bf816381e3e60ae6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:26.373386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a17b5-b056-4916-b5fa-ad3dc8dd867d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:26.375610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 207/431] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:26 2022] GET /identity/v3/projects/c84516b7267f4b43bf816381e3e60ae6 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:26.382048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99383e-ab55-4a79-9f02-6f8ed61a0128 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:26.489845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5cf81-673e-421c-8088-9e76e0407b3f None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:26.493781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5cf81-673e-421c-8088-9e76e0407b3f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:26.494207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5cf81-673e-421c-8088-9e76e0407b3f None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:26.496611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 208/432] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:26.503180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d88a92-dcfc-40f3-aa16-c372f3441001 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:26.503452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d88a92-dcfc-40f3-aa16-c372f3441001 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:26.503870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d88a92-dcfc-40f3-aa16-c372f3441001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:26.525559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-33d88a92-dcfc-40f3-aa16-c372f3441001 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:26.526942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-33d88a92-dcfc-40f3-aa16-c372f3441001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:26.542043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:26.552895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99383e-ab55-4a79-9f02-6f8ed61a0128 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:26.602930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b54d0-204c-467d-8803-f0ffd3dd5c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:26.603580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b54d0-204c-467d-8803-f0ffd3dd5c88 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:26.603854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b54d0-204c-467d-8803-f0ffd3dd5c88 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:26.604124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b54d0-204c-467d-8803-f0ffd3dd5c88 None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:26.612545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b54d0-204c-467d-8803-f0ffd3dd5c88 None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:26.614203 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b54d0-204c-467d-8803-f0ffd3dd5c88 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:26.616110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 225/433] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:26 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:26.616937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b54d0-204c-467d-8803-f0ffd3dd5c88 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:26.618410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:26.639739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 209/434] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:26.641228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d88a92-dcfc-40f3-aa16-c372f3441001 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:26.746989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b54d0-204c-467d-8803-f0ffd3dd5c88 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:26.778507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d88a92-dcfc-40f3-aa16-c372f3441001 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:26.788250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058978d7-b2dc-4caf-b44c-89076c08c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:26.788947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058978d7-b2dc-4caf-b44c-89076c08c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:26.789231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058978d7-b2dc-4caf-b44c-89076c08c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:26.789506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058978d7-b2dc-4caf-b44c-89076c08c8e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:26.802501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058978d7-b2dc-4caf-b44c-89076c08c8e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-058978d7-b2dc-4caf-b44c-89076c08c8e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:26.808099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:26.809127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:26.809935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 226/435] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:05:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:26.815020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058978d7-b2dc-4caf-b44c-89076c08c8e7 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:26.828556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa48be-025b-4c79-a2cf-1f973b4183c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:26.829180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa48be-025b-4c79-a2cf-1f973b4183c1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:26.829459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa48be-025b-4c79-a2cf-1f973b4183c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:26.829729 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa48be-025b-4c79-a2cf-1f973b4183c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:26.830463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa48be-025b-4c79-a2cf-1f973b4183c1 None admin] RBAC: Adding query filter params (group.id=3348dd190d744e649fe289aaee2c6877, role.id=c2fe89b1b9fa47b19e02efd64ace9000, scope.project.id=c84516b7267f4b43bf816381e3e60ae6) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:26.830827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa48be-025b-4c79-a2cf-1f973b4183c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:26.832283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa48be-025b-4c79-a2cf-1f973b4183c1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:26.839787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 210/436] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 22 17:05:26 2022] GET /identity/v3/role_assignments?group.id=3348dd190d744e649fe289aaee2c6877&scope.project.id=c84516b7267f4b43bf816381e3e60ae6&role.id=c2fe89b1b9fa47b19e02efd64ace9000 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:26.852108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:26.868539 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead00cd0-7208-43eb-a84d-e6264b18e7c4 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:26.869165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead00cd0-7208-43eb-a84d-e6264b18e7c4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:26.869637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead00cd0-7208-43eb-a84d-e6264b18e7c4 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:26.871604 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 211/437] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:05:26.878762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aeb570-50cc-4495-877a-1b5d9a609417 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:26.879371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aeb570-50cc-4495-877a-1b5d9a609417 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:26.879860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aeb570-50cc-4495-877a-1b5d9a609417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:26.902907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c0aeb570-50cc-4495-877a-1b5d9a609417 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:26.904653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c0aeb570-50cc-4495-877a-1b5d9a609417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:26.963932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058978d7-b2dc-4caf-b44c-89076c08c8e7 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:26.975637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:27.003213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 212/438] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:27.004278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aeb570-50cc-4495-877a-1b5d9a609417 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:27.006366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93828865-14d1-4b7d-9625-58904ffa5a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:27.007021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93828865-14d1-4b7d-9625-58904ffa5a43 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:27.007322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93828865-14d1-4b7d-9625-58904ffa5a43 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:27.007598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93828865-14d1-4b7d-9625-58904ffa5a43 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:27.008303 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93828865-14d1-4b7d-9625-58904ffa5a43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:27.008826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93828865-14d1-4b7d-9625-58904ffa5a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:27.010278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93828865-14d1-4b7d-9625-58904ffa5a43 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:27.020808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 227/439] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:27.026580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93828865-14d1-4b7d-9625-58904ffa5a43 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:27.157715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:27.159190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aeb570-50cc-4495-877a-1b5d9a609417 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:27.196660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93828865-14d1-4b7d-9625-58904ffa5a43 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:27.201229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f7fb1-b4d1-46af-9224-2cef3d080fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:27.201841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f7fb1-b4d1-46af-9224-2cef3d080fca None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:27.202112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f7fb1-b4d1-46af-9224-2cef3d080fca None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:27.202380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f7fb1-b4d1-46af-9224-2cef3d080fca None admin] PATH_INFO: `/v3/roles/member` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:27.214515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4f7fb1-b4d1-46af-9224-2cef3d080fca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-ff4f7fb1-b4d1-46af-9224-2cef3d080fca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:27.220681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:27.223545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:27.226387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 213/440] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:27.232698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f7fb1-b4d1-46af-9224-2cef3d080fca None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:27.267139 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd26c6-9874-4818-bdb3-b39f60d46baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:27.267679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd26c6-9874-4818-bdb3-b39f60d46baf None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:27.267987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd26c6-9874-4818-bdb3-b39f60d46baf None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:27.268465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd26c6-9874-4818-bdb3-b39f60d46baf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:27.303837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:27.364116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd26c6-9874-4818-bdb3-b39f60d46baf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:27.366654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd26c6-9874-4818-bdb3-b39f60d46baf None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:27.372312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f7fb1-b4d1-46af-9224-2cef3d080fca None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:27.408189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eef03d-c081-40f1-ba48-9d9cebad85e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:27.409369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eef03d-c081-40f1-ba48-9d9cebad85e1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:27.409643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eef03d-c081-40f1-ba48-9d9cebad85e1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:27.409904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eef03d-c081-40f1-ba48-9d9cebad85e1 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:27.410591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eef03d-c081-40f1-ba48-9d9cebad85e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:27.410888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eef03d-c081-40f1-ba48-9d9cebad85e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:27.412284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eef03d-c081-40f1-ba48-9d9cebad85e1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:27.420383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 214/441] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:05:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:27.421177 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eef03d-c081-40f1-ba48-9d9cebad85e1 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:27.446715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:27.467815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 228/442] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 17:05:27.468661 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd26c6-9874-4818-bdb3-b39f60d46baf None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:27.525445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eef03d-c081-40f1-ba48-9d9cebad85e1 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:27.535253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:27.560625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10b6a8-1491-403e-8cd3-479f0f3d9e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:27.561244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10b6a8-1491-403e-8cd3-479f0f3d9e11 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:27.561513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10b6a8-1491-403e-8cd3-479f0f3d9e11 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:27.561775 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10b6a8-1491-403e-8cd3-479f0f3d9e11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:27.572651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10b6a8-1491-403e-8cd3-479f0f3d9e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-fa10b6a8-1491-403e-8cd3-479f0f3d9e11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:27.577783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:27.578743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 215/443] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:05:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:27.580373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10b6a8-1491-403e-8cd3-479f0f3d9e11 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:27.593014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd26c6-9874-4818-bdb3-b39f60d46baf None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:27.631463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575694d-622c-4bb1-97c1-cc6942721fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:27.632158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575694d-622c-4bb1-97c1-cc6942721fe4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:27.632336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575694d-622c-4bb1-97c1-cc6942721fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:27.632605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575694d-622c-4bb1-97c1-cc6942721fe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:27.680451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:27.684076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10b6a8-1491-403e-8cd3-479f0f3d9e11 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:27.701556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575694d-622c-4bb1-97c1-cc6942721fe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:27.703210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575694d-622c-4bb1-97c1-cc6942721fe4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:27.718585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9c14-0e55-4146-8336-eb1f5d0e82c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:27.719240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9c14-0e55-4146-8336-eb1f5d0e82c8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:27.719610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9c14-0e55-4146-8336-eb1f5d0e82c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:27.719885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9c14-0e55-4146-8336-eb1f5d0e82c8 None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:27.721001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e9c14-0e55-4146-8336-eb1f5d0e82c8 None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:27.723117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e9c14-0e55-4146-8336-eb1f5d0e82c8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:27.724760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 216/444] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:27 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:27.725523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9c14-0e55-4146-8336-eb1f5d0e82c8 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:27.764129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:27.804114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 229/445] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:27.804889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575694d-622c-4bb1-97c1-cc6942721fe4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:27.835640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9c14-0e55-4146-8336-eb1f5d0e82c8 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:27.871736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a7223-545f-41bd-9f5b-ea4b20d671f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:27.872347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a7223-545f-41bd-9f5b-ea4b20d671f2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:27.872619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a7223-545f-41bd-9f5b-ea4b20d671f2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:27.872890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a7223-545f-41bd-9f5b-ea4b20d671f2 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:27.873557 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a7223-545f-41bd-9f5b-ea4b20d671f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:27.873860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a7223-545f-41bd-9f5b-ea4b20d671f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:27.875255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a7223-545f-41bd-9f5b-ea4b20d671f2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:27.883980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 217/446] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:27.884850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a7223-545f-41bd-9f5b-ea4b20d671f2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:27.886951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:27.905962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575694d-622c-4bb1-97c1-cc6942721fe4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:27.942376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0baec3-3c99-4bd0-9c90-9002e2b80aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:27.943034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0baec3-3c99-4bd0-9c90-9002e2b80aa5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:27.943307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0baec3-3c99-4bd0-9c90-9002e2b80aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:27.943573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0baec3-3c99-4bd0-9c90-9002e2b80aa5 None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:27.944668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0baec3-3c99-4bd0-9c90-9002e2b80aa5 None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:27.946101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0baec3-3c99-4bd0-9c90-9002e2b80aa5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:27.947697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 230/447] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:27 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:27.948455 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0baec3-3c99-4bd0-9c90-9002e2b80aa5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:27.983244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:27.988109 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a7223-545f-41bd-9f5b-ea4b20d671f2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:28.022176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e1e51-1e2a-4040-8c32-537d599771cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:28.022858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e1e51-1e2a-4040-8c32-537d599771cf None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:28.023131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e1e51-1e2a-4040-8c32-537d599771cf None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:28.023398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e1e51-1e2a-4040-8c32-537d599771cf None admin] PATH_INFO: `/v3/projects/0a33f37b0d0d4ced9878417f46335942` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:28.024505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e1e51-1e2a-4040-8c32-537d599771cf None admin] RBAC: Authorizing `identity:get_project(project_id=0a33f37b0d0d4ced9878417f46335942)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:28.025940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e1e51-1e2a-4040-8c32-537d599771cf None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:28.027628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 218/448] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:27 2022] GET /identity/v3/projects/0a33f37b0d0d4ced9878417f46335942 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:28.028678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e1e51-1e2a-4040-8c32-537d599771cf None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:28.052699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0baec3-3c99-4bd0-9c90-9002e2b80aa5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:28.087408 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bd426-db08-40b9-8200-cdec897d565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:28.087940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bd426-db08-40b9-8200-cdec897d565a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:28.088224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bd426-db08-40b9-8200-cdec897d565a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:28.088500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bd426-db08-40b9-8200-cdec897d565a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:28.107496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:28.135250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e1e51-1e2a-4040-8c32-537d599771cf None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:28.159217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bd426-db08-40b9-8200-cdec897d565a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:28.160728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bd426-db08-40b9-8200-cdec897d565a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:28.170573 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73eedb-c531-4ea6-9a99-6df652dddd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:28.171172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73eedb-c531-4ea6-9a99-6df652dddd84 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:28.171707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73eedb-c531-4ea6-9a99-6df652dddd84 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:28.171979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73eedb-c531-4ea6-9a99-6df652dddd84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:28.172694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73eedb-c531-4ea6-9a99-6df652dddd84 None admin] RBAC: Adding query filter params (group.id=3348dd190d744e649fe289aaee2c6877, role.id=c2fe89b1b9fa47b19e02efd64ace9000, scope.project.id=0a33f37b0d0d4ced9878417f46335942) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:28.172995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73eedb-c531-4ea6-9a99-6df652dddd84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:28.174504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73eedb-c531-4ea6-9a99-6df652dddd84 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:28.180894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 219/449] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 22 17:05:28 2022] GET /identity/v3/role_assignments?group.id=3348dd190d744e649fe289aaee2c6877&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=c2fe89b1b9fa47b19e02efd64ace9000 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:28.181482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73eedb-c531-4ea6-9a99-6df652dddd84 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:28.216390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:28.295562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 231/450] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:28.303410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bd426-db08-40b9-8200-cdec897d565a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:28.314911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73eedb-c531-4ea6-9a99-6df652dddd84 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:28.368136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55c4ca-3079-4407-94d9-29d4c02b17b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:28.371162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55c4ca-3079-4407-94d9-29d4c02b17b2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:28.371162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55c4ca-3079-4407-94d9-29d4c02b17b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:28.371162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55c4ca-3079-4407-94d9-29d4c02b17b2 None admin] PATH_INFO: `/v3/groups/3348dd190d744e649fe289aaee2c6877` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:28.372169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55c4ca-3079-4407-94d9-29d4c02b17b2 None admin] RBAC: Authorizing `identity:get_group(group_id=3348dd190d744e649fe289aaee2c6877)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:28.373667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55c4ca-3079-4407-94d9-29d4c02b17b2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:28.375413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 220/451] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:28 2022] GET /identity/v3/groups/3348dd190d744e649fe289aaee2c6877 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:28.378531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:28.382236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55c4ca-3079-4407-94d9-29d4c02b17b2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:28.438384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bd426-db08-40b9-8200-cdec897d565a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:28.475023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9346299-7fe4-4efd-89e2-13785be16e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:28.475659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346299-7fe4-4efd-89e2-13785be16e5b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:28.475939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346299-7fe4-4efd-89e2-13785be16e5b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:28.476211 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346299-7fe4-4efd-89e2-13785be16e5b None admin] PATH_INFO: `/v3/projects/14e122ee73be44deb3dfe27923205d92` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:28.477334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9346299-7fe4-4efd-89e2-13785be16e5b None admin] RBAC: Authorizing `identity:get_project(project_id=14e122ee73be44deb3dfe27923205d92)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:28.478847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9346299-7fe4-4efd-89e2-13785be16e5b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:28.480511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 232/452] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:05:28 2022] GET /identity/v3/projects/14e122ee73be44deb3dfe27923205d92 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:28.484713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:28.485975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9346299-7fe4-4efd-89e2-13785be16e5b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:28.489290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55c4ca-3079-4407-94d9-29d4c02b17b2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:28.525245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b924703-bad0-4e80-897b-affa889b1fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:28.525964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b924703-bad0-4e80-897b-affa889b1fb5 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:28.526662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b924703-bad0-4e80-897b-affa889b1fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:28.526662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b924703-bad0-4e80-897b-affa889b1fb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:28.527131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b924703-bad0-4e80-897b-affa889b1fb5 None admin] RBAC: Adding query filter params (group.id=3348dd190d744e649fe289aaee2c6877, role.id=13accf89c12040508974db47c6905b31, scope.project.id=0a33f37b0d0d4ced9878417f46335942) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:28.527439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b924703-bad0-4e80-897b-affa889b1fb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:28.528906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b924703-bad0-4e80-897b-affa889b1fb5 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:28.535715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 221/453] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 22 17:05:28 2022] GET /identity/v3/role_assignments?group.id=3348dd190d744e649fe289aaee2c6877&scope.project.id=0a33f37b0d0d4ced9878417f46335942&role.id=13accf89c12040508974db47c6905b31 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:28.602559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9346299-7fe4-4efd-89e2-13785be16e5b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:28.621512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:28.641031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b999af-23ba-4657-8764-8aa24fe78292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:28.641726 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b999af-23ba-4657-8764-8aa24fe78292 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:28.641997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b999af-23ba-4657-8764-8aa24fe78292 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:28.642264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b999af-23ba-4657-8764-8aa24fe78292 None admin] PATH_INFO: `/v3/groups/cba2b2fe80da419d9e2134d21b087500` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:28.644180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b999af-23ba-4657-8764-8aa24fe78292 None admin] RBAC: Authorizing `identity:get_group(group_id=cba2b2fe80da419d9e2134d21b087500)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:28.645635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b999af-23ba-4657-8764-8aa24fe78292 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:28.647333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 233/454] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 22 17:05:28 2022] GET /identity/v3/groups/cba2b2fe80da419d9e2134d21b087500 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:28.653242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b924703-bad0-4e80-897b-affa889b1fb5 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:28.762283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b924703-bad0-4e80-897b-affa889b1fb5 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:28.799000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032dde42-a117-4100-9b12-f7fb83cb35e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:28.800219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dde42-a117-4100-9b12-f7fb83cb35e4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:28.800517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dde42-a117-4100-9b12-f7fb83cb35e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:28.800782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dde42-a117-4100-9b12-f7fb83cb35e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:28.801518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032dde42-a117-4100-9b12-f7fb83cb35e4 None admin] RBAC: Adding query filter params (group.id=cba2b2fe80da419d9e2134d21b087500, role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=14e122ee73be44deb3dfe27923205d92) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:28.801821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032dde42-a117-4100-9b12-f7fb83cb35e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:28.803287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032dde42-a117-4100-9b12-f7fb83cb35e4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:28.809894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 222/455] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 22 17:05:28 2022] GET /identity/v3/role_assignments?group.id=cba2b2fe80da419d9e2134d21b087500&scope.project.id=14e122ee73be44deb3dfe27923205d92&role.id=4b10782d99b74f5bb453b82286012e0c => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:31.151151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee46114-98a6-4dd9-8588-4693ce22d55f None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:31.151669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee46114-98a6-4dd9-8588-4693ce22d55f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:31.151669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee46114-98a6-4dd9-8588-4693ce22d55f None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:31.153283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 234/456] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:31.159662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf656ca-15be-4fa7-b89f-c7c33091a48c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:31.160263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf656ca-15be-4fa7-b89f-c7c33091a48c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:31.160263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf656ca-15be-4fa7-b89f-c7c33091a48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:31.187522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-fdf656ca-15be-4fa7-b89f-c7c33091a48c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:31.188753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fdf656ca-15be-4fa7-b89f-c7c33091a48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:31.218400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:31.289829 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 223/457] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:31.310003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a2dab-ba70-4bd0-b5cf-92f280b0dab4 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:31.310294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a2dab-ba70-4bd0-b5cf-92f280b0dab4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:31.310585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a2dab-ba70-4bd0-b5cf-92f280b0dab4 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:31.312205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 235/458] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:31.318043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393db80-82c9-4f1c-8aa8-1a382e48a40f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:31.318617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393db80-82c9-4f1c-8aa8-1a382e48a40f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:31.319089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393db80-82c9-4f1c-8aa8-1a382e48a40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:31.347856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c393db80-82c9-4f1c-8aa8-1a382e48a40f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:31.349466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c393db80-82c9-4f1c-8aa8-1a382e48a40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:31.452231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 224/459] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:31.638874 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a2dab-ba70-4bd0-b5cf-92f280b0dab4 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:31.786543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a2dab-ba70-4bd0-b5cf-92f280b0dab4 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:31.837103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1b240-53dc-4ddb-b90c-4399eabd4fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:31.837753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1b240-53dc-4ddb-b90c-4399eabd4fbf None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:31.838051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1b240-53dc-4ddb-b90c-4399eabd4fbf None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:31.838307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1b240-53dc-4ddb-b90c-4399eabd4fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:31.844210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431ecd1-d260-4954-b232-10f18b607d2f None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:31.844505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431ecd1-d260-4954-b232-10f18b607d2f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:31.844785 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431ecd1-d260-4954-b232-10f18b607d2f None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:31.847119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 225/460] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:05:31.852433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1b240-53dc-4ddb-b90c-4399eabd4fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:31.852736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c74ad-b33f-4cd7-ad36-4f74f57851c9 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:31.853026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c74ad-b33f-4cd7-ad36-4f74f57851c9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:31.853303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c74ad-b33f-4cd7-ad36-4f74f57851c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:31.853910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1b240-53dc-4ddb-b90c-4399eabd4fbf None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:31.872550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-136c74ad-b33f-4cd7-ad36-4f74f57851c9 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:31.873761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-136c74ad-b33f-4cd7-ad36-4f74f57851c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-71d1b240-53dc-4ddb-b90c-4399eabd4fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:31.874023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:31.874946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:31.875799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:31.875799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:31.875799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:31.875799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 236/461] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:31.876400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1b240-53dc-4ddb-b90c-4399eabd4fbf None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:31.926692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:31.970736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 226/462] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:31.972409 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c74ad-b33f-4cd7-ad36-4f74f57851c9 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:32.005685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:32.012810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1b240-53dc-4ddb-b90c-4399eabd4fbf None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:32.055504 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef6005-e3b3-40b1-942e-3807e2c51fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:32.056469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef6005-e3b3-40b1-942e-3807e2c51fc8 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.057012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef6005-e3b3-40b1-942e-3807e2c51fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.057289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef6005-e3b3-40b1-942e-3807e2c51fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.071495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef6005-e3b3-40b1-942e-3807e2c51fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:32.073730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef6005-e3b3-40b1-942e-3807e2c51fc8 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-05ef6005-e3b3-40b1-942e-3807e2c51fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:32.093827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:32.095673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:32.097694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:32.097694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:32.097694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:32.102502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 237/463] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:32.102502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb6b2f-09f3-440d-a661-8f6f209d95d7 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.102502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb6b2f-09f3-440d-a661-8f6f209d95d7 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.102502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb6b2f-09f3-440d-a661-8f6f209d95d7 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.107206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 238/464] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:32.108106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb6b2f-09f3-440d-a661-8f6f209d95d7 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:32.113378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c74ad-b33f-4cd7-ad36-4f74f57851c9 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:32.156338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8960f-5aa8-44c4-a6c7-5a8e78c14723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:32.157081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8960f-5aa8-44c4-a6c7-5a8e78c14723 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.157424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8960f-5aa8-44c4-a6c7-5a8e78c14723 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.157762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8960f-5aa8-44c4-a6c7-5a8e78c14723 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.159264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8960f-5aa8-44c4-a6c7-5a8e78c14723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:32.159731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8960f-5aa8-44c4-a6c7-5a8e78c14723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:32.161217 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8960f-5aa8-44c4-a6c7-5a8e78c14723 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:32.181031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 227/465] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:32.182445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cc285-b9c9-426d-9bb0-9cc4de83671e None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.182821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cc285-b9c9-426d-9bb0-9cc4de83671e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.183165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cc285-b9c9-426d-9bb0-9cc4de83671e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.203046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-580cc285-b9c9-426d-9bb0-9cc4de83671e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:32.204403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-580cc285-b9c9-426d-9bb0-9cc4de83671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:32.222891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:32.232229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb6b2f-09f3-440d-a661-8f6f209d95d7 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:32.270333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dd580-f666-47d6-93f1-f31c2dd9fbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:32.271155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd580-f666-47d6-93f1-f31c2dd9fbf3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.271502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd580-f666-47d6-93f1-f31c2dd9fbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.271840 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd580-f666-47d6-93f1-f31c2dd9fbf3 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.272571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dd580-f666-47d6-93f1-f31c2dd9fbf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:32.272995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dd580-f666-47d6-93f1-f31c2dd9fbf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:32.274577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dd580-f666-47d6-93f1-f31c2dd9fbf3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:32.286814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 228/466] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:32.287775 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cc285-b9c9-426d-9bb0-9cc4de83671e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:32.291654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 239/467] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:32.293003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dd580-f666-47d6-93f1-f31c2dd9fbf3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:32.297697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:32.413835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dd580-f666-47d6-93f1-f31c2dd9fbf3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:32.422755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cc285-b9c9-426d-9bb0-9cc4de83671e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:32.450413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfcee3-e744-4bea-aed3-ca03e5772f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:32.451326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfcee3-e744-4bea-aed3-ca03e5772f14 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.451742 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfcee3-e744-4bea-aed3-ca03e5772f14 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.452177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfcee3-e744-4bea-aed3-ca03e5772f14 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.452955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfcee3-e744-4bea-aed3-ca03e5772f14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:32.454577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfcee3-e744-4bea-aed3-ca03e5772f14 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:32.462632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79807eb2-057f-4867-b02f-b83aa7f0d71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:32.463250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79807eb2-057f-4867-b02f-b83aa7f0d71c None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.463533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79807eb2-057f-4867-b02f-b83aa7f0d71c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.463806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79807eb2-057f-4867-b02f-b83aa7f0d71c None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.464573 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79807eb2-057f-4867-b02f-b83aa7f0d71c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:32.466118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79807eb2-057f-4867-b02f-b83aa7f0d71c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:32.595579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 240/468] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:05:32 2022] POST /identity/v3/users => generated 248 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:05:32.597313 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfcee3-e744-4bea-aed3-ca03e5772f14 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:32.602043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:32.673825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 229/469] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:05:32 2022] POST /identity/v3/users => generated 246 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:05:32.675670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79807eb2-057f-4867-b02f-b83aa7f0d71c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:32.765605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:32.783544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfcee3-e744-4bea-aed3-ca03e5772f14 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:32.825964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79807eb2-057f-4867-b02f-b83aa7f0d71c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:32.827426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece6fd5-b922-4dba-919e-562b38d5a6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:32.828246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece6fd5-b922-4dba-919e-562b38d5a6d2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.828529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece6fd5-b922-4dba-919e-562b38d5a6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.828799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece6fd5-b922-4dba-919e-562b38d5a6d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.842680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece6fd5-b922-4dba-919e-562b38d5a6d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:32.844182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece6fd5-b922-4dba-919e-562b38d5a6d2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-cece6fd5-b922-4dba-919e-562b38d5a6d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:32.864473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:32.865536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:32.866392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:32.866392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:32.866392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:32.866392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 241/470] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:32.867273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3e310-f073-4d9b-a6d3-db300d6ce07f None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.867546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3e310-f073-4d9b-a6d3-db300d6ce07f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.867815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3e310-f073-4d9b-a6d3-db300d6ce07f None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.870162 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 242/471] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:32.872336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3e310-f073-4d9b-a6d3-db300d6ce07f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:32.884347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31875272-3137-4518-8117-90b37878c713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:32.885083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31875272-3137-4518-8117-90b37878c713 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.885485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31875272-3137-4518-8117-90b37878c713 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.885945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31875272-3137-4518-8117-90b37878c713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.900999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31875272-3137-4518-8117-90b37878c713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:32.902048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31875272-3137-4518-8117-90b37878c713 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-31875272-3137-4518-8117-90b37878c713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:32.923068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:32.924229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:32.925117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:32.925117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:32.925117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:32.925740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 230/472] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:32.930632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d779b8-08a8-4711-9fbb-37011643dbad None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:32.930632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d779b8-08a8-4711-9fbb-37011643dbad None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:32.930632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d779b8-08a8-4711-9fbb-37011643dbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:32.941181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:32.951075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-25d779b8-08a8-4711-9fbb-37011643dbad None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:32.952286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-25d779b8-08a8-4711-9fbb-37011643dbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:33.004283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3e310-f073-4d9b-a6d3-db300d6ce07f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:33.034773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 231/473] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:33.035663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d779b8-08a8-4711-9fbb-37011643dbad None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:33.065086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea133db7-d25f-498f-9619-37ce0e66d7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:33.065704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea133db7-d25f-498f-9619-37ce0e66d7d4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:33.065984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea133db7-d25f-498f-9619-37ce0e66d7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:33.067413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea133db7-d25f-498f-9619-37ce0e66d7d4 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:33.068118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea133db7-d25f-498f-9619-37ce0e66d7d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:33.069562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea133db7-d25f-498f-9619-37ce0e66d7d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:33.073460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea133db7-d25f-498f-9619-37ce0e66d7d4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:33.101677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 243/474] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:33.106590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea133db7-d25f-498f-9619-37ce0e66d7d4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:33.112686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:33.225708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d779b8-08a8-4711-9fbb-37011643dbad None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:33.287903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be94872-5167-43bf-ad3a-baf8217c93c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:33.288522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be94872-5167-43bf-ad3a-baf8217c93c0 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:33.288799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be94872-5167-43bf-ad3a-baf8217c93c0 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:33.289068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be94872-5167-43bf-ad3a-baf8217c93c0 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:33.289716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be94872-5167-43bf-ad3a-baf8217c93c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:33.290030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be94872-5167-43bf-ad3a-baf8217c93c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:33.291468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be94872-5167-43bf-ad3a-baf8217c93c0 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:33.304303 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea133db7-d25f-498f-9619-37ce0e66d7d4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:33.316196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 232/475] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:33.323501 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be94872-5167-43bf-ad3a-baf8217c93c0 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:33.351128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:33.367406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc79be-f058-42fb-b426-93aa23a6687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:33.368946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc79be-f058-42fb-b426-93aa23a6687e None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:33.369234 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc79be-f058-42fb-b426-93aa23a6687e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:33.369509 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc79be-f058-42fb-b426-93aa23a6687e None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:33.370166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc79be-f058-42fb-b426-93aa23a6687e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:33.371681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc79be-f058-42fb-b426-93aa23a6687e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:33.440624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be94872-5167-43bf-ad3a-baf8217c93c0 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:33.444047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:33.477127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0eaf6-54f4-4d6b-b653-4e097aa2e903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:33.477641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0eaf6-54f4-4d6b-b653-4e097aa2e903 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:33.477918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0eaf6-54f4-4d6b-b653-4e097aa2e903 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:33.478185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0eaf6-54f4-4d6b-b653-4e097aa2e903 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:33.478874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0eaf6-54f4-4d6b-b653-4e097aa2e903 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:33.480340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0eaf6-54f4-4d6b-b653-4e097aa2e903 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:33.567932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 244/476] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:05:33 2022] POST /identity/v3/users => generated 249 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:05:33.569176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc79be-f058-42fb-b426-93aa23a6687e None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:33.638322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 233/477] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:05:33 2022] POST /identity/v3/users => generated 248 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:05:33.712547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc79be-f058-42fb-b426-93aa23a6687e None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:33.757657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90716cb8-cc14-4d3b-a106-3478dde7239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:33.758211 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90716cb8-cc14-4d3b-a106-3478dde7239b None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:33.758704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90716cb8-cc14-4d3b-a106-3478dde7239b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:33.758985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90716cb8-cc14-4d3b-a106-3478dde7239b None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:33.759752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90716cb8-cc14-4d3b-a106-3478dde7239b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:33.761312 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90716cb8-cc14-4d3b-a106-3478dde7239b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:33.800273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.sql.core [None req-90716cb8-cc14-4d3b-a106-3478dde7239b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 17:05:33.800273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 22 17:05:33.800273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [parameters: {'id': 'f29d59f37631498aa34858dcbcf8f4f2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 22 17:05:33.800273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82146) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-90716cb8-cc14-4d3b-a106-3478dde7239b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application conn.query(q) Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application result.read() Aug 22 17:05:33.819982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return ref.to_dict() Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 17:05:33.820935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application next(self.gen) Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application yield resource Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application next(self.gen) Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise self.value Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application session.commit() Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 17:05:33.822600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self.session.flush() Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application compat.raise_( Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 17:05:33.823498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 17:05:33.824384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application conn.query(q) Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application result.read() Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 17:05:33.825274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [parameters: {'id': 'f29d59f37631498aa34858dcbcf8f4f2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:33.828750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 22 17:05:33.830821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:33.832798 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 245/478] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 22 17:05:33 2022] POST /identity/v3/roles => generated 144 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:33.832937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0eaf6-54f4-4d6b-b653-4e097aa2e903 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:33.960795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0eaf6-54f4-4d6b-b653-4e097aa2e903 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:33.999448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738e78c-e57f-40a5-8676-0f421f845002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:34.000355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738e78c-e57f-40a5-8676-0f421f845002 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:34.000773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738e78c-e57f-40a5-8676-0f421f845002 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:34.001152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738e78c-e57f-40a5-8676-0f421f845002 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:34.013294 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738e78c-e57f-40a5-8676-0f421f845002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-a738e78c-e57f-40a5-8676-0f421f845002 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:34.018710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:34.019611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:34.020452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 234/479] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:05:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:34.026073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90716cb8-cc14-4d3b-a106-3478dde7239b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:34.067466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:34.137671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90716cb8-cc14-4d3b-a106-3478dde7239b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:34.181522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c235a7-b478-4537-89e0-40d1047a67fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:34.182128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c235a7-b478-4537-89e0-40d1047a67fe None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:34.182404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c235a7-b478-4537-89e0-40d1047a67fe None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:34.182702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c235a7-b478-4537-89e0-40d1047a67fe None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:34.183373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c235a7-b478-4537-89e0-40d1047a67fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:34.183675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c235a7-b478-4537-89e0-40d1047a67fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:34.185054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c235a7-b478-4537-89e0-40d1047a67fe None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:34.197137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 246/480] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:34.878811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d102d8d-dee5-42b6-a6fe-77b5bdd7ef87 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:34.879319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d102d8d-dee5-42b6-a6fe-77b5bdd7ef87 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:34.879319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d102d8d-dee5-42b6-a6fe-77b5bdd7ef87 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:34.880629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 235/481] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:34.886288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a408500-7286-4ab0-91da-06ee86e0848d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:34.886747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a408500-7286-4ab0-91da-06ee86e0848d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:34.886945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a408500-7286-4ab0-91da-06ee86e0848d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:34.906211 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-7a408500-7286-4ab0-91da-06ee86e0848d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:34.907438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7a408500-7286-4ab0-91da-06ee86e0848d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:35.010337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 247/482] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:35.073182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e17ec-ca88-4268-83b1-ba18edfa4568 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:35.073488 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e17ec-ca88-4268-83b1-ba18edfa4568 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:35.073732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e17ec-ca88-4268-83b1-ba18edfa4568 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:35.075137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 236/483] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:35.081307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f90c99-b759-45a2-81ad-48a3fd905f1e None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:35.081594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f90c99-b759-45a2-81ad-48a3fd905f1e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:35.081873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f90c99-b759-45a2-81ad-48a3fd905f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:35.106524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-32f90c99-b759-45a2-81ad-48a3fd905f1e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:35.107724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-32f90c99-b759-45a2-81ad-48a3fd905f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:35.135315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:35.201114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 248/484] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:35.458364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e17ec-ca88-4268-83b1-ba18edfa4568 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:35.482949 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:35.604240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e17ec-ca88-4268-83b1-ba18edfa4568 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:35.644580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f90c99-b759-45a2-81ad-48a3fd905f1e None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:35.652725 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23758dd-e74d-423f-9995-803a86daae32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:35.653366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23758dd-e74d-423f-9995-803a86daae32 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:35.653645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23758dd-e74d-423f-9995-803a86daae32 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:35.653918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23758dd-e74d-423f-9995-803a86daae32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:35.666193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23758dd-e74d-423f-9995-803a86daae32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-f23758dd-e74d-423f-9995-803a86daae32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:35.671542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:35.672423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:35.673207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 237/485] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:35.678387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23758dd-e74d-423f-9995-803a86daae32 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:35.788984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:35.830886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23758dd-e74d-423f-9995-803a86daae32 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:35.876602 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f90c99-b759-45a2-81ad-48a3fd905f1e None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:35.904129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9be536-1d4c-4fee-9485-95e8c7047230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:35.905431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9be536-1d4c-4fee-9485-95e8c7047230 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:35.905907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9be536-1d4c-4fee-9485-95e8c7047230 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:35.906373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9be536-1d4c-4fee-9485-95e8c7047230 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:35.907277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9be536-1d4c-4fee-9485-95e8c7047230 None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:35.907791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9be536-1d4c-4fee-9485-95e8c7047230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:35.909534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9be536-1d4c-4fee-9485-95e8c7047230 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:35.915434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f056f28-06de-4db8-9283-fe6ec87fa7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:35.916076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056f28-06de-4db8-9283-fe6ec87fa7b0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:35.916358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056f28-06de-4db8-9283-fe6ec87fa7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:35.916625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056f28-06de-4db8-9283-fe6ec87fa7b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:35.929082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f056f28-06de-4db8-9283-fe6ec87fa7b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:35.930332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 238/486] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-7f056f28-06de-4db8-9283-fe6ec87fa7b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:35.934457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:35.935391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:35.936183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:35.938284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9be536-1d4c-4fee-9485-95e8c7047230 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:35.940194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 249/487] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:35.942251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f056f28-06de-4db8-9283-fe6ec87fa7b0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:35.964104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:36.056433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f056f28-06de-4db8-9283-fe6ec87fa7b0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:36.093706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c3b27-8203-4e5a-8309-84ccfac9b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:36.094323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c3b27-8203-4e5a-8309-84ccfac9b422 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.094631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c3b27-8203-4e5a-8309-84ccfac9b422 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.094907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c3b27-8203-4e5a-8309-84ccfac9b422 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.096113 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c3b27-8203-4e5a-8309-84ccfac9b422 None admin] RBAC: Adding query filter params (name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:36.096420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c3b27-8203-4e5a-8309-84ccfac9b422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:36.102517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c3b27-8203-4e5a-8309-84ccfac9b422 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:36.112040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 250/488] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:36.113770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c3b27-8203-4e5a-8309-84ccfac9b422 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:36.178785 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9be536-1d4c-4fee-9485-95e8c7047230 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:36.206114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:36.214209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86bdd3-9120-410f-961a-9be971519c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:36.215017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86bdd3-9120-410f-961a-9be971519c1b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.215192 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86bdd3-9120-410f-961a-9be971519c1b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.220182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86bdd3-9120-410f-961a-9be971519c1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.227660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c3b27-8203-4e5a-8309-84ccfac9b422 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:36.228025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86bdd3-9120-410f-961a-9be971519c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:36.229535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86bdd3-9120-410f-961a-9be971519c1b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:36.232076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-3c86bdd3-9120-410f-961a-9be971519c1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:36.246131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:36.247122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:36.247945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:36.247945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:36.247945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:36.247945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 239/489] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:36.248589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152194b-d075-4c2c-9b60-d438548b97ed None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.248770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152194b-d075-4c2c-9b60-d438548b97ed None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.249048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152194b-d075-4c2c-9b60-d438548b97ed None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.254801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 240/490] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:36.256232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e0fb-91a3-4772-8550-380745dbb237 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.256457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e0fb-91a3-4772-8550-380745dbb237 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.256728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e0fb-91a3-4772-8550-380745dbb237 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.262005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 241/491] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:36.262005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0f01a-0699-4b06-918c-99d925a1a7c5 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.262005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0f01a-0699-4b06-918c-99d925a1a7c5 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.262005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0f01a-0699-4b06-918c-99d925a1a7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.275617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85368dc0-4182-4324-92c3-7aab198f3cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:36.276345 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85368dc0-4182-4324-92c3-7aab198f3cba None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.276627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85368dc0-4182-4324-92c3-7aab198f3cba None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.276896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85368dc0-4182-4324-92c3-7aab198f3cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.279426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ecc0f01a-0699-4b06-918c-99d925a1a7c5 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:36.280652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ecc0f01a-0699-4b06-918c-99d925a1a7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:36.288622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85368dc0-4182-4324-92c3-7aab198f3cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:36.290099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85368dc0-4182-4324-92c3-7aab198f3cba None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-85368dc0-4182-4324-92c3-7aab198f3cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:36.305831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:36.306823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:36.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:36.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:36.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:36.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 251/492] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:36.311594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85368dc0-4182-4324-92c3-7aab198f3cba None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:36.367003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 242/493] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:36.368467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a922ba-abb4-4dd6-870c-2dfba3586251 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.368758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a922ba-abb4-4dd6-870c-2dfba3586251 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.369032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a922ba-abb4-4dd6-870c-2dfba3586251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.388049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c1a922ba-abb4-4dd6-870c-2dfba3586251 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:36.389372 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c1a922ba-abb4-4dd6-870c-2dfba3586251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:36.446295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85368dc0-4182-4324-92c3-7aab198f3cba None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:36.465176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 243/494] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:36.466115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a922ba-abb4-4dd6-870c-2dfba3586251 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:36.503693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:36.513258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bf1b0-69e9-47ac-ac1b-4c5d9d97702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:36.513927 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bf1b0-69e9-47ac-ac1b-4c5d9d97702e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.514210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bf1b0-69e9-47ac-ac1b-4c5d9d97702e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.514505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bf1b0-69e9-47ac-ac1b-4c5d9d97702e None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.515220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bf1b0-69e9-47ac-ac1b-4c5d9d97702e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:36.515525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bf1b0-69e9-47ac-ac1b-4c5d9d97702e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:36.517692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bf1b0-69e9-47ac-ac1b-4c5d9d97702e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:36.534621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 252/495] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:36.536525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bf1b0-69e9-47ac-ac1b-4c5d9d97702e None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:36.603790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:36.663453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bf1b0-69e9-47ac-ac1b-4c5d9d97702e None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:36.700222 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626d15d-d877-4e5c-a9fa-31be3d77ece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:36.700862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d15d-d877-4e5c-a9fa-31be3d77ece7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.701141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d15d-d877-4e5c-a9fa-31be3d77ece7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.701414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d15d-d877-4e5c-a9fa-31be3d77ece7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.709615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626d15d-d877-4e5c-a9fa-31be3d77ece7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:36.711111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626d15d-d877-4e5c-a9fa-31be3d77ece7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:36.715193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a922ba-abb4-4dd6-870c-2dfba3586251 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-d626d15d-d877-4e5c-a9fa-31be3d77ece7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:36.728629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:36.729620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:36.730882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 253/496] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:36.731557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626d15d-d877-4e5c-a9fa-31be3d77ece7 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:36.766439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8a755-fce1-47b9-a2f3-15363828d78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:36.767099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8a755-fce1-47b9-a2f3-15363828d78c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.767410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8a755-fce1-47b9-a2f3-15363828d78c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.767687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8a755-fce1-47b9-a2f3-15363828d78c None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.768616 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8a755-fce1-47b9-a2f3-15363828d78c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:36.769856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8a755-fce1-47b9-a2f3-15363828d78c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:36.771351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8a755-fce1-47b9-a2f3-15363828d78c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:36.785061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 244/497] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:36.786388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8a755-fce1-47b9-a2f3-15363828d78c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:36.793807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:36.842335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626d15d-d877-4e5c-a9fa-31be3d77ece7 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:36.878294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29683c3f-9071-4cca-bc48-17e55b0798e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:36.878860 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29683c3f-9071-4cca-bc48-17e55b0798e4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.879134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29683c3f-9071-4cca-bc48-17e55b0798e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.879404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29683c3f-9071-4cca-bc48-17e55b0798e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.889233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8a755-fce1-47b9-a2f3-15363828d78c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:36.889661 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29683c3f-9071-4cca-bc48-17e55b0798e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-29683c3f-9071-4cca-bc48-17e55b0798e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:36.894833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:36.895999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 254/498] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:36.896792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29683c3f-9071-4cca-bc48-17e55b0798e4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:36.904990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:36.926464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7a970-c604-40d0-8b20-d0eec979fff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:36.927097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7a970-c604-40d0-8b20-d0eec979fff6 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:36.927398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7a970-c604-40d0-8b20-d0eec979fff6 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:36.927706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7a970-c604-40d0-8b20-d0eec979fff6 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:36.928441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7a970-c604-40d0-8b20-d0eec979fff6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:36.928747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7a970-c604-40d0-8b20-d0eec979fff6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:36.930199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7a970-c604-40d0-8b20-d0eec979fff6 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:36.966065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 245/499] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 22 17:05:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:36.966937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7a970-c604-40d0-8b20-d0eec979fff6 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:37.002569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29683c3f-9071-4cca-bc48-17e55b0798e4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.015043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:37.037435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba54362-433f-44b6-85e7-2c8f7f5e3fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.038064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba54362-433f-44b6-85e7-2c8f7f5e3fe6 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.038340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba54362-433f-44b6-85e7-2c8f7f5e3fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.038638 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba54362-433f-44b6-85e7-2c8f7f5e3fe6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.050886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba54362-433f-44b6-85e7-2c8f7f5e3fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-4ba54362-433f-44b6-85e7-2c8f7f5e3fe6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:37.056253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:37.057327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:37.058235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 255/500] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:37.063373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf6385-735f-436e-8c45-7a42a07fb606 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.063657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf6385-735f-436e-8c45-7a42a07fb606 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.063929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf6385-735f-436e-8c45-7a42a07fb606 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.065431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 256/501] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:37.066131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf6385-735f-436e-8c45-7a42a07fb606 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:37.072570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7a970-c604-40d0-8b20-d0eec979fff6 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.110345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600028a-eab2-45e9-8918-61a2b5d5aea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.111001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600028a-eab2-45e9-8918-61a2b5d5aea9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.111761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600028a-eab2-45e9-8918-61a2b5d5aea9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.112056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600028a-eab2-45e9-8918-61a2b5d5aea9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.119309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600028a-eab2-45e9-8918-61a2b5d5aea9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:37.120774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600028a-eab2-45e9-8918-61a2b5d5aea9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-d600028a-eab2-45e9-8918-61a2b5d5aea9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:37.133834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:37.135006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:37.135942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 246/502] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 22 17:05:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:37.136522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600028a-eab2-45e9-8918-61a2b5d5aea9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:37.168200 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:37.177817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf6385-735f-436e-8c45-7a42a07fb606 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.211936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d5fd5-090f-4f15-b7bf-259874d1a320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.212469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d5fd5-090f-4f15-b7bf-259874d1a320 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.212741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d5fd5-090f-4f15-b7bf-259874d1a320 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.213007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d5fd5-090f-4f15-b7bf-259874d1a320 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.213664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d5fd5-090f-4f15-b7bf-259874d1a320 None admin] RBAC: Adding query filter params (name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:37.213957 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d5fd5-090f-4f15-b7bf-259874d1a320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:37.215379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d5fd5-090f-4f15-b7bf-259874d1a320 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:37.224543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 257/503] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:37.224543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d5fd5-090f-4f15-b7bf-259874d1a320 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:37.239764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600028a-eab2-45e9-8918-61a2b5d5aea9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.260845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:37.275596 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589996a-cbab-4383-a200-fa6eab6562b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.276340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589996a-cbab-4383-a200-fa6eab6562b3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.276651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589996a-cbab-4383-a200-fa6eab6562b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.276920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589996a-cbab-4383-a200-fa6eab6562b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.332854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d5fd5-090f-4f15-b7bf-259874d1a320 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.355121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589996a-cbab-4383-a200-fa6eab6562b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:37.356614 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589996a-cbab-4383-a200-fa6eab6562b3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:37.370161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c2f28-b60b-4bcd-b7aa-d2fec143333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.370928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c2f28-b60b-4bcd-b7aa-d2fec143333a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.371227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c2f28-b60b-4bcd-b7aa-d2fec143333a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.371501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c2f28-b60b-4bcd-b7aa-d2fec143333a None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.372162 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c2f28-b60b-4bcd-b7aa-d2fec143333a None admin] RBAC: Adding query filter params (name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:37.372459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c2f28-b60b-4bcd-b7aa-d2fec143333a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:37.373919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c2f28-b60b-4bcd-b7aa-d2fec143333a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:37.389597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 258/504] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:37.389597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4612884-3c04-47fc-bd61-306ab07980a2 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.389597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4612884-3c04-47fc-bd61-306ab07980a2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.389597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4612884-3c04-47fc-bd61-306ab07980a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.394615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:37.407150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c4612884-3c04-47fc-bd61-306ab07980a2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:37.408920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c4612884-3c04-47fc-bd61-306ab07980a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:37.471699 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 247/505] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:37.472566 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589996a-cbab-4383-a200-fa6eab6562b3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:37.483042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 259/506] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:37.484110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4612884-3c04-47fc-bd61-306ab07980a2 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:37.516188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:37.590894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589996a-cbab-4383-a200-fa6eab6562b3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.596848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4612884-3c04-47fc-bd61-306ab07980a2 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.623654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:37.630276 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf87b7-89d7-4b59-bc42-88c41945f018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.631045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf87b7-89d7-4b59-bc42-88c41945f018 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.631319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf87b7-89d7-4b59-bc42-88c41945f018 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.631589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf87b7-89d7-4b59-bc42-88c41945f018 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.632316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf87b7-89d7-4b59-bc42-88c41945f018 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:37.632826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf87b7-89d7-4b59-bc42-88c41945f018 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:37.634117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf87b7-89d7-4b59-bc42-88c41945f018 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:37.634283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed15db2-4b2f-4f62-84a0-ca15e1553734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.634912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed15db2-4b2f-4f62-84a0-ca15e1553734 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.635186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed15db2-4b2f-4f62-84a0-ca15e1553734 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.635457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed15db2-4b2f-4f62-84a0-ca15e1553734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.646685 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed15db2-4b2f-4f62-84a0-ca15e1553734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:37.648152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed15db2-4b2f-4f62-84a0-ca15e1553734 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:37.654295 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 248/507] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 22 17:05:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:37.655097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf87b7-89d7-4b59-bc42-88c41945f018 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-8ed15db2-4b2f-4f62-84a0-ca15e1553734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:37.669046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:37.670023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:37.670904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:37.670904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:37.670904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:37.670904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 260/508] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:37.671395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed15db2-4b2f-4f62-84a0-ca15e1553734 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:37.760387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:37.768540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf87b7-89d7-4b59-bc42-88c41945f018 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.782785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed15db2-4b2f-4f62-84a0-ca15e1553734 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.805776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf95f43-3abb-4aa7-b521-84435e5bb014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.806403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf95f43-3abb-4aa7-b521-84435e5bb014 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.806706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf95f43-3abb-4aa7-b521-84435e5bb014 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.806978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf95f43-3abb-4aa7-b521-84435e5bb014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.817940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf95f43-3abb-4aa7-b521-84435e5bb014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:37.818168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b0e84-cccd-4e06-9b04-0117623a5c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.818820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b0e84-cccd-4e06-9b04-0117623a5c7c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.819097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b0e84-cccd-4e06-9b04-0117623a5c7c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.819366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b0e84-cccd-4e06-9b04-0117623a5c7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.819455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf95f43-3abb-4aa7-b521-84435e5bb014 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:37.829814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b0e84-cccd-4e06-9b04-0117623a5c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:37.831305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b0e84-cccd-4e06-9b04-0117623a5c7c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-abf95f43-3abb-4aa7-b521-84435e5bb014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:37.834012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:37.835099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:37.835964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:37.835964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:37.835964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:37.835964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 249/509] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:37.835964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf95f43-3abb-4aa7-b521-84435e5bb014 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-832b0e84-cccd-4e06-9b04-0117623a5c7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:37.845391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:37.846603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:37.847850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:37.847850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:37.847850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:37.847850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 261/510] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:37.850315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b0e84-cccd-4e06-9b04-0117623a5c7c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:37.885262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:37.943721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf95f43-3abb-4aa7-b521-84435e5bb014 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.950870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b0e84-cccd-4e06-9b04-0117623a5c7c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:37.976784 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a7f7a-af30-49fb-8204-2eb982020282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.977387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a7f7a-af30-49fb-8204-2eb982020282 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.977667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a7f7a-af30-49fb-8204-2eb982020282 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.977935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a7f7a-af30-49fb-8204-2eb982020282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.978858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:37.984471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57064d-db68-4d85-ab02-900f1116f417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:37.985074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57064d-db68-4d85-ab02-900f1116f417 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:37.985351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57064d-db68-4d85-ab02-900f1116f417 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:37.985619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57064d-db68-4d85-ab02-900f1116f417 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:37.986965 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57064d-db68-4d85-ab02-900f1116f417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:37.986965 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57064d-db68-4d85-ab02-900f1116f417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:37.988616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57064d-db68-4d85-ab02-900f1116f417 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.000148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 262/511] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:38.000957 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57064d-db68-4d85-ab02-900f1116f417 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.047932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a7f7a-af30-49fb-8204-2eb982020282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.049929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a7f7a-af30-49fb-8204-2eb982020282 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.092320 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:38.100335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57064d-db68-4d85-ab02-900f1116f417 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:38.135378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644864f1-6d94-487b-903f-ee637d7ae283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:38.136015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644864f1-6d94-487b-903f-ee637d7ae283 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:38.136281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644864f1-6d94-487b-903f-ee637d7ae283 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:38.136548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644864f1-6d94-487b-903f-ee637d7ae283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:38.146929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644864f1-6d94-487b-903f-ee637d7ae283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.148388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644864f1-6d94-487b-903f-ee637d7ae283 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.150843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 250/512] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:38.151605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a7f7a-af30-49fb-8204-2eb982020282 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-644864f1-6d94-487b-903f-ee637d7ae283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:38.163293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:38.164223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:38.165074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:38.165074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:38.165074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:38.165074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 263/513] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:38.165297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644864f1-6d94-487b-903f-ee637d7ae283 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.210029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:38.255731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a7f7a-af30-49fb-8204-2eb982020282 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:38.266903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644864f1-6d94-487b-903f-ee637d7ae283 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:38.291055 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0c965-aeea-4755-8dcf-528ae72ff7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:38.291651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0c965-aeea-4755-8dcf-528ae72ff7b3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:38.291925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0c965-aeea-4755-8dcf-528ae72ff7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:38.292201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0c965-aeea-4755-8dcf-528ae72ff7b3 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:38.292828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0c965-aeea-4755-8dcf-528ae72ff7b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:38.293128 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0c965-aeea-4755-8dcf-528ae72ff7b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.294513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0c965-aeea-4755-8dcf-528ae72ff7b3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.302572 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f6976-6cd2-4b42-885b-b1a83499155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:38.303187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f6976-6cd2-4b42-885b-b1a83499155f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:38.303468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f6976-6cd2-4b42-885b-b1a83499155f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:38.303751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f6976-6cd2-4b42-885b-b1a83499155f None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:38.304839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f6976-6cd2-4b42-885b-b1a83499155f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:38.305149 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f6976-6cd2-4b42-885b-b1a83499155f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.306546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f6976-6cd2-4b42-885b-b1a83499155f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.306919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 251/514] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:38.307728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0c965-aeea-4755-8dcf-528ae72ff7b3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.310217 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:38.322061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 264/515] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:38.325605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f6976-6cd2-4b42-885b-b1a83499155f None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.415639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0c965-aeea-4755-8dcf-528ae72ff7b3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:38.433893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f6976-6cd2-4b42-885b-b1a83499155f None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:38.435455 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:38.450265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034a0e3-c6c0-4afa-abbd-7d587b9cd059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:38.450915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034a0e3-c6c0-4afa-abbd-7d587b9cd059 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:38.451192 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034a0e3-c6c0-4afa-abbd-7d587b9cd059 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:38.451457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034a0e3-c6c0-4afa-abbd-7d587b9cd059 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:38.459215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034a0e3-c6c0-4afa-abbd-7d587b9cd059 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.460731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034a0e3-c6c0-4afa-abbd-7d587b9cd059 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.469488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62da4-c83f-4ff7-a654-7bb280a1bdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:38.470119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62da4-c83f-4ff7-a654-7bb280a1bdf1 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:38.471074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62da4-c83f-4ff7-a654-7bb280a1bdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:38.471465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62da4-c83f-4ff7-a654-7bb280a1bdf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-e034a0e3-c6c0-4afa-abbd-7d587b9cd059 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:38.472524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:38.473654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 252/516] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:38.474650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034a0e3-c6c0-4afa-abbd-7d587b9cd059 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.487145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e62da4-c83f-4ff7-a654-7bb280a1bdf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.488653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e62da4-c83f-4ff7-a654-7bb280a1bdf1 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-e0e62da4-c83f-4ff7-a654-7bb280a1bdf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:38.506603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:38.507689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:38.508551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:38.508551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:38.508551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:38.508551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 265/517] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:38.508551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62da4-c83f-4ff7-a654-7bb280a1bdf1 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.543102 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:38.583066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034a0e3-c6c0-4afa-abbd-7d587b9cd059 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:38.612049 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62da4-c83f-4ff7-a654-7bb280a1bdf1 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:38.617926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99912ff-62cd-44e7-a412-c14a3b50a4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:38.618573 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99912ff-62cd-44e7-a412-c14a3b50a4b4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:38.618853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99912ff-62cd-44e7-a412-c14a3b50a4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:38.619123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99912ff-62cd-44e7-a412-c14a3b50a4b4 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:38.619765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99912ff-62cd-44e7-a412-c14a3b50a4b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:38.620064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99912ff-62cd-44e7-a412-c14a3b50a4b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.621446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99912ff-62cd-44e7-a412-c14a3b50a4b4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.635832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 253/518] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:38.636807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99912ff-62cd-44e7-a412-c14a3b50a4b4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.652395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7399f57-46a5-4d86-b652-d3881f99780d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:38.653021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7399f57-46a5-4d86-b652-d3881f99780d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:38.653299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7399f57-46a5-4d86-b652-d3881f99780d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:38.653566 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7399f57-46a5-4d86-b652-d3881f99780d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:38.661636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7399f57-46a5-4d86-b652-d3881f99780d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.663124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7399f57-46a5-4d86-b652-d3881f99780d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-a7399f57-46a5-4d86-b652-d3881f99780d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:38.674635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:38.675657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:38.676716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:38.676716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:38.676716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:38.676716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:38.676716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:38.676716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:38.676716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:38.676716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:38.676716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 17:05:38.676716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:38.678628 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 266/519] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:38.679786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7399f57-46a5-4d86-b652-d3881f99780d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.695182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:38.738331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99912ff-62cd-44e7-a412-c14a3b50a4b4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:38.760936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:38.772820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9b7e4-a031-4bd5-8819-55888793e8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:38.773449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9b7e4-a031-4bd5-8819-55888793e8c1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:38.773728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9b7e4-a031-4bd5-8819-55888793e8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:38.773999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9b7e4-a031-4bd5-8819-55888793e8c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:38.780732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9b7e4-a031-4bd5-8819-55888793e8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.782178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9b7e4-a031-4bd5-8819-55888793e8c1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.782573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7399f57-46a5-4d86-b652-d3881f99780d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-69e9b7e4-a031-4bd5-8819-55888793e8c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:38.795328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:38.796605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:38.802461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:38.802461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:38.802461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:38.802461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:38.802461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:38.802461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:38.802461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:38.802461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:38.802461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 254/520] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:38.802461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9b7e4-a031-4bd5-8819-55888793e8c1 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.823786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546682e-bf64-4c67-97fc-84cd2d4e9d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:38.824470 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546682e-bf64-4c67-97fc-84cd2d4e9d49 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:38.824751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546682e-bf64-4c67-97fc-84cd2d4e9d49 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:38.825052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546682e-bf64-4c67-97fc-84cd2d4e9d49 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:38.825798 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546682e-bf64-4c67-97fc-84cd2d4e9d49 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:38.826512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546682e-bf64-4c67-97fc-84cd2d4e9d49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.830516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546682e-bf64-4c67-97fc-84cd2d4e9d49 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.870666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 267/521] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:38.870666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546682e-bf64-4c67-97fc-84cd2d4e9d49 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.917123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9b7e4-a031-4bd5-8819-55888793e8c1 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:38.953326 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffdc10-b043-44f6-a38b-c65b7f0aea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:38.953993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffdc10-b043-44f6-a38b-c65b7f0aea57 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:38.954197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffdc10-b043-44f6-a38b-c65b7f0aea57 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:38.954460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffdc10-b043-44f6-a38b-c65b7f0aea57 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:38.955844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffdc10-b043-44f6-a38b-c65b7f0aea57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:38.956067 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffdc10-b043-44f6-a38b-c65b7f0aea57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:38.957472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffdc10-b043-44f6-a38b-c65b7f0aea57 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:38.959539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:38.971854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 255/522] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:38.977620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffdc10-b043-44f6-a38b-c65b7f0aea57 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:38.991123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546682e-bf64-4c67-97fc-84cd2d4e9d49 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:39.024349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:39.029849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34f555-a22e-4ca5-8868-3f594ca84730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:39.030509 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34f555-a22e-4ca5-8868-3f594ca84730 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:39.030806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34f555-a22e-4ca5-8868-3f594ca84730 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:39.031083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34f555-a22e-4ca5-8868-3f594ca84730 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:39.031734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34f555-a22e-4ca5-8868-3f594ca84730 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:39.034214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34f555-a22e-4ca5-8868-3f594ca84730 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:39.085265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffdc10-b043-44f6-a38b-c65b7f0aea57 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:39.120787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c11f4b-3ecb-439b-bf6e-de7b0c5f033c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:39.121390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11f4b-3ecb-439b-bf6e-de7b0c5f033c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:39.121669 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11f4b-3ecb-439b-bf6e-de7b0c5f033c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:39.121970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11f4b-3ecb-439b-bf6e-de7b0c5f033c None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:39.122744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c11f4b-3ecb-439b-bf6e-de7b0c5f033c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:39.123051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c11f4b-3ecb-439b-bf6e-de7b0c5f033c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:39.124497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c11f4b-3ecb-439b-bf6e-de7b0c5f033c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:39.141167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 256/523] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 22 17:05:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:39.141967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c11f4b-3ecb-439b-bf6e-de7b0c5f033c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:39.197398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 268/524] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:05:38 2022] POST /identity/v3/users => generated 247 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:05:39.198293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34f555-a22e-4ca5-8868-3f594ca84730 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:39.259358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:39.263605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c11f4b-3ecb-439b-bf6e-de7b0c5f033c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:39.304521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b6748-d988-4545-9f1a-d2e0884062b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:39.305250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b6748-d988-4545-9f1a-d2e0884062b2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:39.305526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b6748-d988-4545-9f1a-d2e0884062b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:39.305919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b6748-d988-4545-9f1a-d2e0884062b2 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:39.306673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b6748-d988-4545-9f1a-d2e0884062b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:39.306982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b6748-d988-4545-9f1a-d2e0884062b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:39.308543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b6748-d988-4545-9f1a-d2e0884062b2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:39.322117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34f555-a22e-4ca5-8868-3f594ca84730 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:39.335364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 257/525] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:39.335364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b6748-d988-4545-9f1a-d2e0884062b2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:39.362972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:39.371843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9441-1836-4b46-bdd8-2778efd1f026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:39.372647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9441-1836-4b46-bdd8-2778efd1f026 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:39.372927 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9441-1836-4b46-bdd8-2778efd1f026 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:39.373200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9441-1836-4b46-bdd8-2778efd1f026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:39.452244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b6748-d988-4545-9f1a-d2e0884062b2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:39.462069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce9441-1836-4b46-bdd8-2778efd1f026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:39.463966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce9441-1836-4b46-bdd8-2778efd1f026 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:39.489759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908235e9-277e-46b6-9098-a39f98f662e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:39.491049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908235e9-277e-46b6-9098-a39f98f662e0 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:39.491049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908235e9-277e-46b6-9098-a39f98f662e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:39.491049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908235e9-277e-46b6-9098-a39f98f662e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:39.498180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908235e9-277e-46b6-9098-a39f98f662e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:39.499725 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908235e9-277e-46b6-9098-a39f98f662e0 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-908235e9-277e-46b6-9098-a39f98f662e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:39.511962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:39.512845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:39.513733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:39.513733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:39.513733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:39.513733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:39.513733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:39.513733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:39.513733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:39.513733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:39.513733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 258/526] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:39.514197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908235e9-277e-46b6-9098-a39f98f662e0 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:39.544600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:39.581395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 269/527] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:39.582270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9441-1836-4b46-bdd8-2778efd1f026 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:39.615053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:39.636239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908235e9-277e-46b6-9098-a39f98f662e0 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:39.676311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9a61f-2b2a-4fb0-a490-7f290aa87efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:39.676943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9a61f-2b2a-4fb0-a490-7f290aa87efb None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:39.677224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9a61f-2b2a-4fb0-a490-7f290aa87efb None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:39.677497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9a61f-2b2a-4fb0-a490-7f290aa87efb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:39.695199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9441-1836-4b46-bdd8-2778efd1f026 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:39.733739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa6784-239c-4a18-8bd9-b5287f782469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:39.734645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa6784-239c-4a18-8bd9-b5287f782469 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:39.734967 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa6784-239c-4a18-8bd9-b5287f782469 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:39.735182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa6784-239c-4a18-8bd9-b5287f782469 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:39.736002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa6784-239c-4a18-8bd9-b5287f782469 None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=ea4c2870e2754f3188228fc441950df2) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:39.736871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa6784-239c-4a18-8bd9-b5287f782469 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:39.738310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa6784-239c-4a18-8bd9-b5287f782469 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:39.754507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 270/528] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:39 2022] GET /identity/v3/role_assignments?user.id=ea4c2870e2754f3188228fc441950df2&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:39.754507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa6784-239c-4a18-8bd9-b5287f782469 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:39.767640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9a61f-2b2a-4fb0-a490-7f290aa87efb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:39.769300 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9a61f-2b2a-4fb0-a490-7f290aa87efb None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:39.794394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:39.884768 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:39.896684 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa6784-239c-4a18-8bd9-b5287f782469 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:39.897773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 259/529] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:39.898837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9a61f-2b2a-4fb0-a490-7f290aa87efb None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:39.957107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6920b-870a-494e-998a-4b7adce18e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:39.958172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6920b-870a-494e-998a-4b7adce18e68 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:39.959123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6920b-870a-494e-998a-4b7adce18e68 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:39.959503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6920b-870a-494e-998a-4b7adce18e68 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:39.960590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6920b-870a-494e-998a-4b7adce18e68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:39.960996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6920b-870a-494e-998a-4b7adce18e68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:39.963143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6920b-870a-494e-998a-4b7adce18e68 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:39.988839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 271/530] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:39.989814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6920b-870a-494e-998a-4b7adce18e68 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:40.029377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9a61f-2b2a-4fb0-a490-7f290aa87efb None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:40.055479 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:40.072266 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a054436-a2f6-426b-9365-2a9195abac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:40.073313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a054436-a2f6-426b-9365-2a9195abac72 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:40.073857 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a054436-a2f6-426b-9365-2a9195abac72 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:40.074340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a054436-a2f6-426b-9365-2a9195abac72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:40.087162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a054436-a2f6-426b-9365-2a9195abac72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:40.088923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a054436-a2f6-426b-9365-2a9195abac72 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-2a054436-a2f6-426b-9365-2a9195abac72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:40.105624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:40.107244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:40.109204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:40.109204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:40.109204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:40.113275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 260/531] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:40.114189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a054436-a2f6-426b-9365-2a9195abac72 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:40.140779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6920b-870a-494e-998a-4b7adce18e68 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:40.207943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea11fd-4db1-4a56-97a6-f18a3648ccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:40.208706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea11fd-4db1-4a56-97a6-f18a3648ccd8 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:40.208914 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea11fd-4db1-4a56-97a6-f18a3648ccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:40.214502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea11fd-4db1-4a56-97a6-f18a3648ccd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:40.220759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea11fd-4db1-4a56-97a6-f18a3648ccd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:40.222348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea11fd-4db1-4a56-97a6-f18a3648ccd8 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-cdea11fd-4db1-4a56-97a6-f18a3648ccd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:40.240392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:40.242567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:40.244662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:40.244662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:40.244662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:40.244662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 272/532] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:40.244662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea11fd-4db1-4a56-97a6-f18a3648ccd8 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:40.250907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:40.279548 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a054436-a2f6-426b-9365-2a9195abac72 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:40.342458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67923d-64f5-4939-96c7-3f2ebff208a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:40.343104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67923d-64f5-4939-96c7-3f2ebff208a6 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:40.343384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67923d-64f5-4939-96c7-3f2ebff208a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:40.343661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67923d-64f5-4939-96c7-3f2ebff208a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:40.344403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67923d-64f5-4939-96c7-3f2ebff208a6 None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=9628118c1df64fa4a7ea8ce37549d8b3) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:40.344735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67923d-64f5-4939-96c7-3f2ebff208a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:40.346958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67923d-64f5-4939-96c7-3f2ebff208a6 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:40.353079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 261/533] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:40 2022] GET /identity/v3/role_assignments?user.id=9628118c1df64fa4a7ea8ce37549d8b3&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:40.353786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67923d-64f5-4939-96c7-3f2ebff208a6 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:40.373631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea11fd-4db1-4a56-97a6-f18a3648ccd8 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:40.399414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:40.454365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf467c6-5054-4d48-8291-d5cb73616824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:40.456149 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf467c6-5054-4d48-8291-d5cb73616824 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:40.456767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf467c6-5054-4d48-8291-d5cb73616824 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:40.457287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf467c6-5054-4d48-8291-d5cb73616824 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:40.458234 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf467c6-5054-4d48-8291-d5cb73616824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:40.458946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf467c6-5054-4d48-8291-d5cb73616824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:40.468342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf467c6-5054-4d48-8291-d5cb73616824 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:40.475717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67923d-64f5-4939-96c7-3f2ebff208a6 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:40.492537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 273/534] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:40.498444 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf467c6-5054-4d48-8291-d5cb73616824 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:40.517008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda8b2f-c79e-4693-9ead-55f067a2a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:40.517008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda8b2f-c79e-4693-9ead-55f067a2a92c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:40.517008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda8b2f-c79e-4693-9ead-55f067a2a92c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:40.517008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda8b2f-c79e-4693-9ead-55f067a2a92c None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:40.518011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda8b2f-c79e-4693-9ead-55f067a2a92c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:40.518011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda8b2f-c79e-4693-9ead-55f067a2a92c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:40.518574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda8b2f-c79e-4693-9ead-55f067a2a92c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:40.531514 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 262/535] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:40.537207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda8b2f-c79e-4693-9ead-55f067a2a92c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:40.646890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:40.653994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda8b2f-c79e-4693-9ead-55f067a2a92c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:40.675552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:40.699533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ba411-790a-4365-89c3-5c6b6ed4104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:40.700174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ba411-790a-4365-89c3-5c6b6ed4104a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:40.700453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ba411-790a-4365-89c3-5c6b6ed4104a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:40.700722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ba411-790a-4365-89c3-5c6b6ed4104a None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:40.712179 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ba411-790a-4365-89c3-5c6b6ed4104a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:40.713782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ba411-790a-4365-89c3-5c6b6ed4104a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-107ba411-790a-4365-89c3-5c6b6ed4104a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:40.734461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:40.735405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:40.736315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:40.736315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:40.736315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:40.736315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:40.736315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:40.736315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:40.736315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:40.736315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:40.740235 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 263/536] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:40.748399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ba411-790a-4365-89c3-5c6b6ed4104a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:40.757962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf467c6-5054-4d48-8291-d5cb73616824 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:40.809022 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3485e9-7354-4306-b139-258372aa7498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:40.809612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3485e9-7354-4306-b139-258372aa7498 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:40.809890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3485e9-7354-4306-b139-258372aa7498 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:40.810161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3485e9-7354-4306-b139-258372aa7498 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:40.819735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3485e9-7354-4306-b139-258372aa7498 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:40.821570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3485e9-7354-4306-b139-258372aa7498 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-9f3485e9-7354-4306-b139-258372aa7498 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:40.836797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:40.837716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:40.838606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:40.838606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:40.838606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:40.838606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:40.838606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:40.838606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:40.838606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:40.838606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:40.838606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 274/537] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:40.843716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3485e9-7354-4306-b139-258372aa7498 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:40.960650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ba411-790a-4365-89c3-5c6b6ed4104a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:40.999601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c8dbb-e510-4849-bea8-0a8383f61dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:41.000207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c8dbb-e510-4849-bea8-0a8383f61dc7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:41.000483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c8dbb-e510-4849-bea8-0a8383f61dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:41.000754 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c8dbb-e510-4849-bea8-0a8383f61dc7 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:41.001500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c8dbb-e510-4849-bea8-0a8383f61dc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:41.002334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c8dbb-e510-4849-bea8-0a8383f61dc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:41.003773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c8dbb-e510-4849-bea8-0a8383f61dc7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:41.017891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 264/538] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:41.020197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3485e9-7354-4306-b139-258372aa7498 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:41.024541 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c8dbb-e510-4849-bea8-0a8383f61dc7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:41.046436 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:41.076173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:41.101235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5295792-2f27-47a4-bb25-ed0b81b8b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:41.102200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5295792-2f27-47a4-bb25-ed0b81b8b85d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:41.102682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5295792-2f27-47a4-bb25-ed0b81b8b85d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:41.103131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5295792-2f27-47a4-bb25-ed0b81b8b85d None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:41.104012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5295792-2f27-47a4-bb25-ed0b81b8b85d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:41.104523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5295792-2f27-47a4-bb25-ed0b81b8b85d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:41.106195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5295792-2f27-47a4-bb25-ed0b81b8b85d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:41.122937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 275/539] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:41.136019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5295792-2f27-47a4-bb25-ed0b81b8b85d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:41.185659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c8dbb-e510-4849-bea8-0a8383f61dc7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:41.226115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ef1ad-fdc3-47b1-b85a-f92247cbd4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:41.226907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ef1ad-fdc3-47b1-b85a-f92247cbd4d7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:41.227282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ef1ad-fdc3-47b1-b85a-f92247cbd4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:41.227695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ef1ad-fdc3-47b1-b85a-f92247cbd4d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:41.229113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ef1ad-fdc3-47b1-b85a-f92247cbd4d7 None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=68088e59a2c64ff2948e0549f1fc7cb9) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:41.229520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ef1ad-fdc3-47b1-b85a-f92247cbd4d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:41.231098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ef1ad-fdc3-47b1-b85a-f92247cbd4d7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:41.238791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 265/540] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:41 2022] GET /identity/v3/role_assignments?user.id=68088e59a2c64ff2948e0549f1fc7cb9&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:41.316114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5295792-2f27-47a4-bb25-ed0b81b8b85d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:41.356642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37902978-a561-4472-ba8a-6b17b3682853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:41.357173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37902978-a561-4472-ba8a-6b17b3682853 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:41.357452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37902978-a561-4472-ba8a-6b17b3682853 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:41.357727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37902978-a561-4472-ba8a-6b17b3682853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:41.358505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37902978-a561-4472-ba8a-6b17b3682853 None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=c29a91e099ba4cd5886fd910d978fb4d) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:41.358840 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37902978-a561-4472-ba8a-6b17b3682853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:41.360324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37902978-a561-4472-ba8a-6b17b3682853 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:41.367986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 276/541] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:41 2022] GET /identity/v3/role_assignments?user.id=c29a91e099ba4cd5886fd910d978fb4d&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:41.505970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bb770-9a0a-41d0-a6df-ce0832781b80 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:41.506339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bb770-9a0a-41d0-a6df-ce0832781b80 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:41.506574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bb770-9a0a-41d0-a6df-ce0832781b80 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:41.508174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 266/542] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:41.513850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e71a89-9760-4567-99a2-4dbaee3ee2cd None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:41.514160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e71a89-9760-4567-99a2-4dbaee3ee2cd None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:41.514451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e71a89-9760-4567-99a2-4dbaee3ee2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:41.540636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-97e71a89-9760-4567-99a2-4dbaee3ee2cd None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:41.541910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-97e71a89-9760-4567-99a2-4dbaee3ee2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:41.611649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:41.640895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 277/543] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:42.029630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bb770-9a0a-41d0-a6df-ce0832781b80 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:42.152847 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:42.156601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b05828-f4a2-469d-8c73-5510d81a40c4 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:42.157009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b05828-f4a2-469d-8c73-5510d81a40c4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:42.157384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b05828-f4a2-469d-8c73-5510d81a40c4 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:42.159783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 278/544] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:42.165623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfe9b4-3eae-40de-b95b-43bb42d843b4 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:42.171195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfe9b4-3eae-40de-b95b-43bb42d843b4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:42.171195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfe9b4-3eae-40de-b95b-43bb42d843b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:42.179743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bb770-9a0a-41d0-a6df-ce0832781b80 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:42.199481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-fbcfe9b4-3eae-40de-b95b-43bb42d843b4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:42.200680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fbcfe9b4-3eae-40de-b95b-43bb42d843b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:42.221172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b78b14-8ab5-4451-8e69-b4c75184eedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:42.223090 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b78b14-8ab5-4451-8e69-b4c75184eedb None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:42.223369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b78b14-8ab5-4451-8e69-b4c75184eedb None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:42.223638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b78b14-8ab5-4451-8e69-b4c75184eedb None admin] PATH_INFO: `/v3/roles/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:42.245370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b78b14-8ab5-4451-8e69-b4c75184eedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-c0b78b14-8ab5-4451-8e69-b4c75184eedb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:42.254386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:42.255351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:42.256163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:42.257472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 267/545] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:42.265808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b78b14-8ab5-4451-8e69-b4c75184eedb None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:42.335068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 279/546] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:42.427889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b78b14-8ab5-4451-8e69-b4c75184eedb None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:42.480986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b586649-90d4-4357-96ec-5f5b3e933dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:42.481684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b586649-90d4-4357-96ec-5f5b3e933dca None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:42.481998 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b586649-90d4-4357-96ec-5f5b3e933dca None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:42.482306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b586649-90d4-4357-96ec-5f5b3e933dca None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:42.483089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b586649-90d4-4357-96ec-5f5b3e933dca None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:42.483393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b586649-90d4-4357-96ec-5f5b3e933dca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:42.484778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b586649-90d4-4357-96ec-5f5b3e933dca None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:42.500832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 268/547] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:42.506973 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfe9b4-3eae-40de-b95b-43bb42d843b4 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:42.597245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87d93c-96e2-4d7f-8382-d405b6284c6f None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:42.597534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87d93c-96e2-4d7f-8382-d405b6284c6f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:42.597716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87d93c-96e2-4d7f-8382-d405b6284c6f None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:42.599127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 269/548] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:42.617683 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335c5c-4d27-437e-8367-56ae5a2f787b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:42.622610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335c5c-4d27-437e-8367-56ae5a2f787b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:42.622892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335c5c-4d27-437e-8367-56ae5a2f787b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:42.635494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:42.657282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ef335c5c-4d27-437e-8367-56ae5a2f787b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:42.660159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ef335c5c-4d27-437e-8367-56ae5a2f787b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:42.681665 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfe9b4-3eae-40de-b95b-43bb42d843b4 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:42.700219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:42.734784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e7bdb-456a-43f0-ab23-b9ef529bec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:42.735506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e7bdb-456a-43f0-ab23-b9ef529bec1c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:42.736905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e7bdb-456a-43f0-ab23-b9ef529bec1c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:42.736905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e7bdb-456a-43f0-ab23-b9ef529bec1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:42.748712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e7bdb-456a-43f0-ab23-b9ef529bec1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:42.750291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e7bdb-456a-43f0-ab23-b9ef529bec1c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-2b7e7bdb-456a-43f0-ab23-b9ef529bec1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:42.769417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:42.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:42.772272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:42.772272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:42.772272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:42.772272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 280/549] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:42.772272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e7bdb-456a-43f0-ab23-b9ef529bec1c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:42.778635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 270/550] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:42.779730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335c5c-4d27-437e-8367-56ae5a2f787b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:42.932875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335c5c-4d27-437e-8367-56ae5a2f787b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:42.979751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:42.986800 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2deb6-041c-491c-a591-3f77d96d3a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:42.987944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2deb6-041c-491c-a591-3f77d96d3a7f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:42.996026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2deb6-041c-491c-a591-3f77d96d3a7f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:42.996026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2deb6-041c-491c-a591-3f77d96d3a7f None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:42.996026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2deb6-041c-491c-a591-3f77d96d3a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:42.996026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2deb6-041c-491c-a591-3f77d96d3a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:42.996026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2deb6-041c-491c-a591-3f77d96d3a7f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:42.996530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e7bdb-456a-43f0-ab23-b9ef529bec1c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:43.010784 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 271/551] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:43.015984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2deb6-041c-491c-a591-3f77d96d3a7f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:43.072586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e6fe3-6e80-42e5-b0f8-e4a5d6d2f2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:43.073232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e6fe3-6e80-42e5-b0f8-e4a5d6d2f2b0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.073513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e6fe3-6e80-42e5-b0f8-e4a5d6d2f2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.073788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e6fe3-6e80-42e5-b0f8-e4a5d6d2f2b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.096396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e6fe3-6e80-42e5-b0f8-e4a5d6d2f2b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-1b6e6fe3-6e80-42e5-b0f8-e4a5d6d2f2b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:43.101534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:43.102508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:43.103330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 281/552] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:43.104206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e6fe3-6e80-42e5-b0f8-e4a5d6d2f2b0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:43.133882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:43.140593 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2deb6-041c-491c-a591-3f77d96d3a7f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:43.180375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c17835-2419-4429-806e-9fe632d97acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:43.181031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c17835-2419-4429-806e-9fe632d97acd None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.181310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c17835-2419-4429-806e-9fe632d97acd None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.181681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c17835-2419-4429-806e-9fe632d97acd None admin] PATH_INFO: `/v3/users/swift` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.190905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c17835-2419-4429-806e-9fe632d97acd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:43.192428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c17835-2419-4429-806e-9fe632d97acd None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-95c17835-2419-4429-806e-9fe632d97acd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:43.206136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:43.207134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:43.208062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 272/553] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:05:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:43.209142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c17835-2419-4429-806e-9fe632d97acd None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:43.218292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e6fe3-6e80-42e5-b0f8-e4a5d6d2f2b0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:43.256800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918f3c3-e7e0-4b45-a4c6-69596bd91e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:43.257437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918f3c3-e7e0-4b45-a4c6-69596bd91e74 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.257712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918f3c3-e7e0-4b45-a4c6-69596bd91e74 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.257983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918f3c3-e7e0-4b45-a4c6-69596bd91e74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.269488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918f3c3-e7e0-4b45-a4c6-69596bd91e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:43.271809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-c918f3c3-e7e0-4b45-a4c6-69596bd91e74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:43.274268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:43.276803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 282/554] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:43.278806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918f3c3-e7e0-4b45-a4c6-69596bd91e74 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:43.334299 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c17835-2419-4429-806e-9fe632d97acd None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:43.363251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:43.377016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8e660-61a4-4a81-8802-299233288c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:43.378029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8e660-61a4-4a81-8802-299233288c22 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.378311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8e660-61a4-4a81-8802-299233288c22 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.378609 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8e660-61a4-4a81-8802-299233288c22 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.379282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8e660-61a4-4a81-8802-299233288c22 None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:43.379580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8e660-61a4-4a81-8802-299233288c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:43.380940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8e660-61a4-4a81-8802-299233288c22 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:43.383398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918f3c3-e7e0-4b45-a4c6-69596bd91e74 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:43.389738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 273/555] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:43.390695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8e660-61a4-4a81-8802-299233288c22 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:43.426116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26255502-8a66-4c60-8298-cb94fd9b46f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:43.426930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26255502-8a66-4c60-8298-cb94fd9b46f4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.427211 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26255502-8a66-4c60-8298-cb94fd9b46f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.427777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26255502-8a66-4c60-8298-cb94fd9b46f4 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.428794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26255502-8a66-4c60-8298-cb94fd9b46f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:43.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26255502-8a66-4c60-8298-cb94fd9b46f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:43.430800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26255502-8a66-4c60-8298-cb94fd9b46f4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:43.454689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 283/556] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 22 17:05:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:43.455847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26255502-8a66-4c60-8298-cb94fd9b46f4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:43.507000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8e660-61a4-4a81-8802-299233288c22 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:43.526626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:43.550816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059939a-c0d1-400b-b183-80fec6a48b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:43.551428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059939a-c0d1-400b-b183-80fec6a48b59 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.551703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059939a-c0d1-400b-b183-80fec6a48b59 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.551972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059939a-c0d1-400b-b183-80fec6a48b59 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.552637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059939a-c0d1-400b-b183-80fec6a48b59 None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:43.552958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059939a-c0d1-400b-b183-80fec6a48b59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:43.554374 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059939a-c0d1-400b-b183-80fec6a48b59 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:43.568538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 274/557] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:43.569462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059939a-c0d1-400b-b183-80fec6a48b59 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:43.590963 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26255502-8a66-4c60-8298-cb94fd9b46f4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:43.634241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91786db8-637f-4428-b004-a1be77bb9065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:43.635031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91786db8-637f-4428-b004-a1be77bb9065 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.635388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91786db8-637f-4428-b004-a1be77bb9065 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.635720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91786db8-637f-4428-b004-a1be77bb9065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.644340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:43.648793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91786db8-637f-4428-b004-a1be77bb9065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:43.650567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91786db8-637f-4428-b004-a1be77bb9065 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-91786db8-637f-4428-b004-a1be77bb9065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:43.669351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:43.671155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:43.672053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:43.672053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:43.672053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:43.672053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 284/558] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:43.672053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91786db8-637f-4428-b004-a1be77bb9065 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:43.702525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059939a-c0d1-400b-b183-80fec6a48b59 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:43.738567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1d142-a4e3-48f7-8a3b-b80aff221ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:43.740900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1d142-a4e3-48f7-8a3b-b80aff221ae2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.741183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1d142-a4e3-48f7-8a3b-b80aff221ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.741454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1d142-a4e3-48f7-8a3b-b80aff221ae2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.778951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91786db8-637f-4428-b004-a1be77bb9065 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:43.787242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:43.810640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1d142-a4e3-48f7-8a3b-b80aff221ae2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:43.812232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1d142-a4e3-48f7-8a3b-b80aff221ae2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:43.815048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3d025-b8c5-4e7a-8d37-0768a8e18cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:43.815677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3d025-b8c5-4e7a-8d37-0768a8e18cf5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.815951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3d025-b8c5-4e7a-8d37-0768a8e18cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.816219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3d025-b8c5-4e7a-8d37-0768a8e18cf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.826720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3d025-b8c5-4e7a-8d37-0768a8e18cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:43.828162 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3d025-b8c5-4e7a-8d37-0768a8e18cf5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-6aa3d025-b8c5-4e7a-8d37-0768a8e18cf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:43.842915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:43.844216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:43.845759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:43.845759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:43.845759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:43.845759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 285/559] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:43.845759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ee7d7-f043-4fcc-83aa-8949d68a24f1 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.846039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ee7d7-f043-4fcc-83aa-8949d68a24f1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.846548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ee7d7-f043-4fcc-83aa-8949d68a24f1 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.847486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 286/560] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:43.848181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ee7d7-f043-4fcc-83aa-8949d68a24f1 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:43.893849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:43.918967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 275/561] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:43.920225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee63a46-3061-412f-b72a-32e79a2826db None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.920511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee63a46-3061-412f-b72a-32e79a2826db None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.921036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee63a46-3061-412f-b72a-32e79a2826db None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.922282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 276/562] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:43.925406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee63a46-3061-412f-b72a-32e79a2826db None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:43.953698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ee7d7-f043-4fcc-83aa-8949d68a24f1 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:43.988023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d9811-6190-4e78-9eac-a33ff4f15327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:43.988684 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d9811-6190-4e78-9eac-a33ff4f15327 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:43.988907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d9811-6190-4e78-9eac-a33ff4f15327 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:43.989178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d9811-6190-4e78-9eac-a33ff4f15327 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:43.989815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d9811-6190-4e78-9eac-a33ff4f15327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:43.990116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d9811-6190-4e78-9eac-a33ff4f15327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:43.991534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d9811-6190-4e78-9eac-a33ff4f15327 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:44.004613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:44.004987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 287/563] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:44.006194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671eec47-bb9f-42d6-b512-e73b8db387e8 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.006474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671eec47-bb9f-42d6-b512-e73b8db387e8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.006775 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671eec47-bb9f-42d6-b512-e73b8db387e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.024269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-671eec47-bb9f-42d6-b512-e73b8db387e8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:44.025521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-671eec47-bb9f-42d6-b512-e73b8db387e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:44.033635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee63a46-3061-412f-b72a-32e79a2826db None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:44.070626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4a5c7-f01e-4eb3-a0f5-52b6190f26fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:44.071231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4a5c7-f01e-4eb3-a0f5-52b6190f26fd None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.071505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4a5c7-f01e-4eb3-a0f5-52b6190f26fd None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.071771 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4a5c7-f01e-4eb3-a0f5-52b6190f26fd None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.072409 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4a5c7-f01e-4eb3-a0f5-52b6190f26fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:44.072715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4a5c7-f01e-4eb3-a0f5-52b6190f26fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:44.074078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4a5c7-f01e-4eb3-a0f5-52b6190f26fd None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:44.085459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 277/564] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:44.087367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4a5c7-f01e-4eb3-a0f5-52b6190f26fd None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:44.092906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 288/565] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:44.094145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de918f81-4d76-4bfe-a777-2513a2f40e2a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.094435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de918f81-4d76-4bfe-a777-2513a2f40e2a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.094742 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de918f81-4d76-4bfe-a777-2513a2f40e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.123469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-de918f81-4d76-4bfe-a777-2513a2f40e2a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:44.124849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-de918f81-4d76-4bfe-a777-2513a2f40e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:44.154472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:44.194105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4a5c7-f01e-4eb3-a0f5-52b6190f26fd None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:44.203868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 289/566] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:44.204615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de918f81-4d76-4bfe-a777-2513a2f40e2a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:44.243384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbe248-72af-4aa7-805c-620a3f7fcafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:44.244602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbe248-72af-4aa7-805c-620a3f7fcafc None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.244891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbe248-72af-4aa7-805c-620a3f7fcafc None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.245440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbe248-72af-4aa7-805c-620a3f7fcafc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.249936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:44.260757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbe248-72af-4aa7-805c-620a3f7fcafc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:44.262881 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbe248-72af-4aa7-805c-620a3f7fcafc None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-c1cbe248-72af-4aa7-805c-620a3f7fcafc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:44.283721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:44.284698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:44.285590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:44.285590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:44.285590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:44.285590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 278/567] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:44.286552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbe248-72af-4aa7-805c-620a3f7fcafc None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:44.338069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de918f81-4d76-4bfe-a777-2513a2f40e2a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:44.384121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeceb34-7538-43b2-8328-c6acf01c2cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:44.385555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeceb34-7538-43b2-8328-c6acf01c2cf4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.385839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeceb34-7538-43b2-8328-c6acf01c2cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.386115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeceb34-7538-43b2-8328-c6acf01c2cf4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.395170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeceb34-7538-43b2-8328-c6acf01c2cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:44.397305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeceb34-7538-43b2-8328-c6acf01c2cf4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-cbeceb34-7538-43b2-8328-c6acf01c2cf4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:44.412640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:44.413655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:44.414627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:44.414627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:44.414627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:44.414627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:44.414627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:44.414627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:44.414627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:44.414627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:44.414627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 17:05:44.414627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:44.415189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 290/568] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:44.415758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeceb34-7538-43b2-8328-c6acf01c2cf4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:44.416431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbe248-72af-4aa7-805c-620a3f7fcafc None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:44.444648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:44.454738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb2f5f-f642-4a2c-9d79-8860ffaae306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:44.455274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb2f5f-f642-4a2c-9d79-8860ffaae306 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.455558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb2f5f-f642-4a2c-9d79-8860ffaae306 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.455817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb2f5f-f642-4a2c-9d79-8860ffaae306 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.463683 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb2f5f-f642-4a2c-9d79-8860ffaae306 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:44.465138 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb2f5f-f642-4a2c-9d79-8860ffaae306 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-8cfb2f5f-f642-4a2c-9d79-8860ffaae306 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:44.477752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:44.479121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 279/569] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:44.480317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb2f5f-f642-4a2c-9d79-8860ffaae306 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:44.529688 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeceb34-7538-43b2-8328-c6acf01c2cf4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:44.534397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:44.563609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91660941-6f78-4888-b750-93ed4d54a93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:44.564461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91660941-6f78-4888-b750-93ed4d54a93a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.564742 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91660941-6f78-4888-b750-93ed4d54a93a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.565016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91660941-6f78-4888-b750-93ed4d54a93a None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.565659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91660941-6f78-4888-b750-93ed4d54a93a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:44.565961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91660941-6f78-4888-b750-93ed4d54a93a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:44.567383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91660941-6f78-4888-b750-93ed4d54a93a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:44.579120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 291/570] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:44.580497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91660941-6f78-4888-b750-93ed4d54a93a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:44.588836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb2f5f-f642-4a2c-9d79-8860ffaae306 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:44.624635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcba4a5-8e1e-4ec9-bd84-a29cef068227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:44.625267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcba4a5-8e1e-4ec9-bd84-a29cef068227 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.625543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcba4a5-8e1e-4ec9-bd84-a29cef068227 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.625813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcba4a5-8e1e-4ec9-bd84-a29cef068227 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.636929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcba4a5-8e1e-4ec9-bd84-a29cef068227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-7fcba4a5-8e1e-4ec9-bd84-a29cef068227 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:44.642314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:44.643315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:44.644146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 280/571] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:44.644841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcba4a5-8e1e-4ec9-bd84-a29cef068227 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:44.671210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:44.689563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91660941-6f78-4888-b750-93ed4d54a93a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:44.725359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6ebf-c397-444e-b97b-a5400ef789ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:44.725962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6ebf-c397-444e-b97b-a5400ef789ac None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.726242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6ebf-c397-444e-b97b-a5400ef789ac None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.726555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6ebf-c397-444e-b97b-a5400ef789ac None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.727290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce6ebf-c397-444e-b97b-a5400ef789ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:44.727593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce6ebf-c397-444e-b97b-a5400ef789ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:44.729045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce6ebf-c397-444e-b97b-a5400ef789ac None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:44.746859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 292/572] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:44.747694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6ebf-c397-444e-b97b-a5400ef789ac None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:44.759266 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcba4a5-8e1e-4ec9-bd84-a29cef068227 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:44.768319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:44.793606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635bd52-af1c-4624-aff3-fb1ce0ccedb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:44.794152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635bd52-af1c-4624-aff3-fb1ce0ccedb3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.794428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635bd52-af1c-4624-aff3-fb1ce0ccedb3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.794747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635bd52-af1c-4624-aff3-fb1ce0ccedb3 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.795482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635bd52-af1c-4624-aff3-fb1ce0ccedb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:44.796039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635bd52-af1c-4624-aff3-fb1ce0ccedb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:44.797477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635bd52-af1c-4624-aff3-fb1ce0ccedb3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:44.816630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 281/573] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:44.817455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635bd52-af1c-4624-aff3-fb1ce0ccedb3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:44.857436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6ebf-c397-444e-b97b-a5400ef789ac None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:44.891922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52f9f9-efc0-4a21-aa60-eb505878c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:44.892866 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52f9f9-efc0-4a21-aa60-eb505878c996 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.893143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52f9f9-efc0-4a21-aa60-eb505878c996 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.893413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52f9f9-efc0-4a21-aa60-eb505878c996 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.900816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:44.903729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52f9f9-efc0-4a21-aa60-eb505878c996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-6a52f9f9-efc0-4a21-aa60-eb505878c996 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:44.908717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:44.909672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:44.910530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 293/574] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:44.911244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52f9f9-efc0-4a21-aa60-eb505878c996 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:44.920201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635bd52-af1c-4624-aff3-fb1ce0ccedb3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:44.956264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df760d-e135-40a3-9c83-07db777ba8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:44.956885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df760d-e135-40a3-9c83-07db777ba8d5 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:44.957169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df760d-e135-40a3-9c83-07db777ba8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:44.957439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df760d-e135-40a3-9c83-07db777ba8d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:44.964181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df760d-e135-40a3-9c83-07db777ba8d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:44.965636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df760d-e135-40a3-9c83-07db777ba8d5 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-15df760d-e135-40a3-9c83-07db777ba8d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:44.977413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:44.978418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:44.979331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:44.979331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:44.979331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:44.979331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:44.979331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:44.979331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:44.979331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:44.979331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:44.979331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 282/575] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:44.979817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df760d-e135-40a3-9c83-07db777ba8d5 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.006719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:45.021355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52f9f9-efc0-4a21-aa60-eb505878c996 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.059018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219fb82-a014-4115-9221-eb6ff12d8853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.059857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219fb82-a014-4115-9221-eb6ff12d8853 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.060283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219fb82-a014-4115-9221-eb6ff12d8853 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.060554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219fb82-a014-4115-9221-eb6ff12d8853 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.061220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219fb82-a014-4115-9221-eb6ff12d8853 None admin] RBAC: Adding query filter params (name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:45.061527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219fb82-a014-4115-9221-eb6ff12d8853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.062929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219fb82-a014-4115-9221-eb6ff12d8853 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.071227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 294/576] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:45.072106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219fb82-a014-4115-9221-eb6ff12d8853 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.081681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df760d-e135-40a3-9c83-07db777ba8d5 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.117214 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ed3df-f7bf-4c2a-bf95-fcde02befac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.117819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ed3df-f7bf-4c2a-bf95-fcde02befac3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.118089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ed3df-f7bf-4c2a-bf95-fcde02befac3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.118354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ed3df-f7bf-4c2a-bf95-fcde02befac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.128535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ed3df-f7bf-4c2a-bf95-fcde02befac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.129993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ed3df-f7bf-4c2a-bf95-fcde02befac3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.134852 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-d35ed3df-f7bf-4c2a-bf95-fcde02befac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:45.144243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:45.149925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:45.151101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:45.151101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:45.151101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:45.151101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 283/577] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:45.151101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ed3df-f7bf-4c2a-bf95-fcde02befac3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.185640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219fb82-a014-4115-9221-eb6ff12d8853 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.218443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8dce-3ddd-474d-a6e1-d915429f70c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.219099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8dce-3ddd-474d-a6e1-d915429f70c9 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.219377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8dce-3ddd-474d-a6e1-d915429f70c9 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.219649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8dce-3ddd-474d-a6e1-d915429f70c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.225928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:45.230878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c8dce-3ddd-474d-a6e1-d915429f70c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.232342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c8dce-3ddd-474d-a6e1-d915429f70c9 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-269c8dce-3ddd-474d-a6e1-d915429f70c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:45.247156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:45.248118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:45.249011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:45.249011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:45.249011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:45.249188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ed3df-f7bf-4c2a-bf95-fcde02befac3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.249263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 295/578] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:45.249263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8dce-3ddd-474d-a6e1-d915429f70c9 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.283250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360f1e6-af44-4a56-bc9e-49da87b650b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.283944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360f1e6-af44-4a56-bc9e-49da87b650b1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.284249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360f1e6-af44-4a56-bc9e-49da87b650b1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.284519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360f1e6-af44-4a56-bc9e-49da87b650b1 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.285183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360f1e6-af44-4a56-bc9e-49da87b650b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:45.285483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360f1e6-af44-4a56-bc9e-49da87b650b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.286909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360f1e6-af44-4a56-bc9e-49da87b650b1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.294906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 284/579] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:45.295953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360f1e6-af44-4a56-bc9e-49da87b650b1 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.358301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8dce-3ddd-474d-a6e1-d915429f70c9 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.372379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:45.393482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db547278-e74e-4110-a0e0-a8fb9825d90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.394152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db547278-e74e-4110-a0e0-a8fb9825d90c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.394436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db547278-e74e-4110-a0e0-a8fb9825d90c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.394738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db547278-e74e-4110-a0e0-a8fb9825d90c None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.395431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db547278-e74e-4110-a0e0-a8fb9825d90c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:45.395743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db547278-e74e-4110-a0e0-a8fb9825d90c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.397317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db547278-e74e-4110-a0e0-a8fb9825d90c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.404192 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360f1e6-af44-4a56-bc9e-49da87b650b1 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.410706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 296/580] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:45.411546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db547278-e74e-4110-a0e0-a8fb9825d90c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.443190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd25bc2-5a2a-4583-aeb0-e537acd0d25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.443835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd25bc2-5a2a-4583-aeb0-e537acd0d25d None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.444129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd25bc2-5a2a-4583-aeb0-e537acd0d25d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.444408 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd25bc2-5a2a-4583-aeb0-e537acd0d25d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.453098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:45.455724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd25bc2-5a2a-4583-aeb0-e537acd0d25d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.457186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd25bc2-5a2a-4583-aeb0-e537acd0d25d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-7bd25bc2-5a2a-4583-aeb0-e537acd0d25d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:45.471720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:45.472687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:45.473658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:45.473658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:45.473658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:45.473658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 285/581] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:45.473658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd25bc2-5a2a-4583-aeb0-e537acd0d25d None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.519581 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db547278-e74e-4110-a0e0-a8fb9825d90c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.553697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c12db4-bcd6-4f9b-8eb5-dc3e7fc4dd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.554295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c12db4-bcd6-4f9b-8eb5-dc3e7fc4dd0a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.554609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c12db4-bcd6-4f9b-8eb5-dc3e7fc4dd0a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.554883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c12db4-bcd6-4f9b-8eb5-dc3e7fc4dd0a None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.555522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c12db4-bcd6-4f9b-8eb5-dc3e7fc4dd0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:45.555820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c12db4-bcd6-4f9b-8eb5-dc3e7fc4dd0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.557179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c12db4-bcd6-4f9b-8eb5-dc3e7fc4dd0a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.568280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 297/582] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:45.569099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c12db4-bcd6-4f9b-8eb5-dc3e7fc4dd0a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.578962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd25bc2-5a2a-4583-aeb0-e537acd0d25d None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.604962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:45.613837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66cbb2-6319-4895-8a51-033054c941ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.614378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66cbb2-6319-4895-8a51-033054c941ef None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.614681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66cbb2-6319-4895-8a51-033054c941ef None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.614951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66cbb2-6319-4895-8a51-033054c941ef None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.615587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66cbb2-6319-4895-8a51-033054c941ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:45.615885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66cbb2-6319-4895-8a51-033054c941ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.617257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66cbb2-6319-4895-8a51-033054c941ef None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.630663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 286/583] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:45.631503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66cbb2-6319-4895-8a51-033054c941ef None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.673181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c12db4-bcd6-4f9b-8eb5-dc3e7fc4dd0a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.685794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:45.708129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a336a7-db15-4e63-a957-021d45c93e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.708766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a336a7-db15-4e63-a957-021d45c93e78 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.709039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a336a7-db15-4e63-a957-021d45c93e78 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.709307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a336a7-db15-4e63-a957-021d45c93e78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.720609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a336a7-db15-4e63-a957-021d45c93e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.722100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a336a7-db15-4e63-a957-021d45c93e78 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.734878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66cbb2-6319-4895-8a51-033054c941ef None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-74a336a7-db15-4e63-a957-021d45c93e78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:45.736147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:45.737268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:45.738196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:45.738196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:45.738196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:45.738196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 298/584] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:45.738196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a336a7-db15-4e63-a957-021d45c93e78 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.776847 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f510a-bcb3-4147-b570-747a191c636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.777439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f510a-bcb3-4147-b570-747a191c636c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.777711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f510a-bcb3-4147-b570-747a191c636c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.777977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f510a-bcb3-4147-b570-747a191c636c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.778722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f510a-bcb3-4147-b570-747a191c636c None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=38462b1165514ed7b948cba710782d4b) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:45.779026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f510a-bcb3-4147-b570-747a191c636c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.780481 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f510a-bcb3-4147-b570-747a191c636c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.786207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 287/585] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/role_assignments?user.id=38462b1165514ed7b948cba710782d4b&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:45.787102 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f510a-bcb3-4147-b570-747a191c636c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.833322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:45.852820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a336a7-db15-4e63-a957-021d45c93e78 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.895608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d9ea4-8033-4752-ab94-7496ca424c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.896132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d9ea4-8033-4752-ab94-7496ca424c83 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.896408 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d9ea4-8033-4752-ab94-7496ca424c83 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.896682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d9ea4-8033-4752-ab94-7496ca424c83 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.897333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d9ea4-8033-4752-ab94-7496ca424c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:45.897633 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d9ea4-8033-4752-ab94-7496ca424c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.899079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d9ea4-8033-4752-ab94-7496ca424c83 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.917374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 299/586] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:45.917374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d9ea4-8033-4752-ab94-7496ca424c83 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:45.930424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f510a-bcb3-4147-b570-747a191c636c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:45.963467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:45.968269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e35f4-e311-4400-8248-be194b373d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:45.968903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e35f4-e311-4400-8248-be194b373d06 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:45.969228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e35f4-e311-4400-8248-be194b373d06 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:45.976875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e35f4-e311-4400-8248-be194b373d06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:45.978804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e35f4-e311-4400-8248-be194b373d06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:45.980426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e35f4-e311-4400-8248-be194b373d06 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-dc3e35f4-e311-4400-8248-be194b373d06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:45.992851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:45.993766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:45.994866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:45.994866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:45.994866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:45.994866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:45.994866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:45.994866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:45.994866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:45.994866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:45.996807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 288/587] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:45.997861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e35f4-e311-4400-8248-be194b373d06 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:46.027878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d9ea4-8033-4752-ab94-7496ca424c83 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:46.064595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e10d51-0ba9-49c4-9cdf-6fbdd393387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:46.065237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e10d51-0ba9-49c4-9cdf-6fbdd393387d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:46.065512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e10d51-0ba9-49c4-9cdf-6fbdd393387d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:46.065783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e10d51-0ba9-49c4-9cdf-6fbdd393387d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:46.072869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e10d51-0ba9-49c4-9cdf-6fbdd393387d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:46.074338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e10d51-0ba9-49c4-9cdf-6fbdd393387d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-83e10d51-0ba9-49c4-9cdf-6fbdd393387d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:46.086825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:46.087821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:46.088709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:46.088709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:46.088709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:46.088709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:46.088709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:46.088709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:46.088709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:46.088709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:46.089785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 300/588] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:46.090649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e10d51-0ba9-49c4-9cdf-6fbdd393387d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:46.100148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:46.117814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e35f4-e311-4400-8248-be194b373d06 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:46.154171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf708a-32d6-4347-9dcd-297a57f34b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:46.154817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf708a-32d6-4347-9dcd-297a57f34b96 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:46.155096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf708a-32d6-4347-9dcd-297a57f34b96 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:46.155369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf708a-32d6-4347-9dcd-297a57f34b96 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:46.156014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf708a-32d6-4347-9dcd-297a57f34b96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:46.156316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf708a-32d6-4347-9dcd-297a57f34b96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:46.158537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf708a-32d6-4347-9dcd-297a57f34b96 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:46.175986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 289/589] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:46.175986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf708a-32d6-4347-9dcd-297a57f34b96 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:46.210666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:46.213428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e10d51-0ba9-49c4-9cdf-6fbdd393387d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:46.251548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6dad-d2f9-4a85-a143-d8e0e6406ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:46.252427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6dad-d2f9-4a85-a143-d8e0e6406ea4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:46.252871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6dad-d2f9-4a85-a143-d8e0e6406ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:46.253302 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6dad-d2f9-4a85-a143-d8e0e6406ea4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:46.261773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a6dad-d2f9-4a85-a143-d8e0e6406ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:46.263448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a6dad-d2f9-4a85-a143-d8e0e6406ea4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ef6a6dad-d2f9-4a85-a143-d8e0e6406ea4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:46.275447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:46.276609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:46.277759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:46.277759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:46.277759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:46.277759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:46.277759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:46.277759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:46.277759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:46.277759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:46.277759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 17:05:46.277759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:46.279563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 301/590] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:46.280477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6dad-d2f9-4a85-a143-d8e0e6406ea4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:46.288397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf708a-32d6-4347-9dcd-297a57f34b96 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:46.332261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3d737-9145-4bce-9248-2ec99f163913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:46.333184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3d737-9145-4bce-9248-2ec99f163913 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:46.333677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3d737-9145-4bce-9248-2ec99f163913 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:46.334091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3d737-9145-4bce-9248-2ec99f163913 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:46.335014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3d737-9145-4bce-9248-2ec99f163913 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:46.338236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3d737-9145-4bce-9248-2ec99f163913 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:46.338236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3d737-9145-4bce-9248-2ec99f163913 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:46.351645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 290/591] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:46.352702 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3d737-9145-4bce-9248-2ec99f163913 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:46.353259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:46.394107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6dad-d2f9-4a85-a143-d8e0e6406ea4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:46.437143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96b411-20e4-4a5c-ae73-df81bfe9eed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:46.437672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96b411-20e4-4a5c-ae73-df81bfe9eed2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:46.437968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96b411-20e4-4a5c-ae73-df81bfe9eed2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:46.439856 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96b411-20e4-4a5c-ae73-df81bfe9eed2 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:46.440422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96b411-20e4-4a5c-ae73-df81bfe9eed2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:46.440811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96b411-20e4-4a5c-ae73-df81bfe9eed2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:46.442348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96b411-20e4-4a5c-ae73-df81bfe9eed2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:46.459953 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 302/592] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:46.459953 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96b411-20e4-4a5c-ae73-df81bfe9eed2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:46.464957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:46.471373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3d737-9145-4bce-9248-2ec99f163913 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:46.512301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7563-778c-4e88-a26b-398de77f87d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:46.513034 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7563-778c-4e88-a26b-398de77f87d8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:46.513398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7563-778c-4e88-a26b-398de77f87d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:46.513737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7563-778c-4e88-a26b-398de77f87d8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:46.521865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f7563-778c-4e88-a26b-398de77f87d8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:46.523438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f7563-778c-4e88-a26b-398de77f87d8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-6e1f7563-778c-4e88-a26b-398de77f87d8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:46.536414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:46.537563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:46.538762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:46.538762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:46.538762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:46.538762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:46.538762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:46.538762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:46.538762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:46.538762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:46.538762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 17:05:46.538762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:46.540615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 291/593] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:46.541521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7563-778c-4e88-a26b-398de77f87d8 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:46.584406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96b411-20e4-4a5c-ae73-df81bfe9eed2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:46.615397 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:46.629263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07326d-71c6-4fda-afd4-f8ce27bc5e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:46.641782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07326d-71c6-4fda-afd4-f8ce27bc5e90 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:46.641782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07326d-71c6-4fda-afd4-f8ce27bc5e90 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:46.641782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07326d-71c6-4fda-afd4-f8ce27bc5e90 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:46.641782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07326d-71c6-4fda-afd4-f8ce27bc5e90 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:46.641782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07326d-71c6-4fda-afd4-f8ce27bc5e90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:46.641782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07326d-71c6-4fda-afd4-f8ce27bc5e90 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:46.653709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 303/594] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 22 17:05:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:46.655033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07326d-71c6-4fda-afd4-f8ce27bc5e90 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:46.658462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7563-778c-4e88-a26b-398de77f87d8 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:46.705964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abb835-dae0-4fbc-ba56-d3a2486f07a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:46.707088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abb835-dae0-4fbc-ba56-d3a2486f07a7 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:46.707512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abb835-dae0-4fbc-ba56-d3a2486f07a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:46.707987 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abb835-dae0-4fbc-ba56-d3a2486f07a7 None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/ea4c2870e2754f3188228fc441950df2/roles/14a34ac79fe64a978eca3e5f9089e72f` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:46.725156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abb835-dae0-4fbc-ba56-d3a2486f07a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=ea4c2870e2754f3188228fc441950df2, role_id=14a34ac79fe64a978eca3e5f9089e72f)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:46.727027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abb835-dae0-4fbc-ba56-d3a2486f07a7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:46.766428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07326d-71c6-4fda-afd4-f8ce27bc5e90 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:46.792715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 292/595] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:05:46 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/ea4c2870e2754f3188228fc441950df2/roles/14a34ac79fe64a978eca3e5f9089e72f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:05:46.796198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abb835-dae0-4fbc-ba56-d3a2486f07a7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:46.810711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:46.822512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e950abe-7826-4d3a-b24d-bd32a87e1a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:46.823714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e950abe-7826-4d3a-b24d-bd32a87e1a10 None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:46.824097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e950abe-7826-4d3a-b24d-bd32a87e1a10 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:46.824490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e950abe-7826-4d3a-b24d-bd32a87e1a10 None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/9628118c1df64fa4a7ea8ce37549d8b3/roles/14a34ac79fe64a978eca3e5f9089e72f` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:46.846099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e950abe-7826-4d3a-b24d-bd32a87e1a10 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=9628118c1df64fa4a7ea8ce37549d8b3, role_id=14a34ac79fe64a978eca3e5f9089e72f)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:46.848826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e950abe-7826-4d3a-b24d-bd32a87e1a10 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:46.933946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 304/596] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:05:46 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/9628118c1df64fa4a7ea8ce37549d8b3/roles/14a34ac79fe64a978eca3e5f9089e72f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:05:46.935521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e950abe-7826-4d3a-b24d-bd32a87e1a10 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:46.943443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abb835-dae0-4fbc-ba56-d3a2486f07a7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:46.997552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268659b-ff0c-4167-b1b5-f85384c1362f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:46.998848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:46.999468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268659b-ff0c-4167-b1b5-f85384c1362f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:46.999468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268659b-ff0c-4167-b1b5-f85384c1362f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:46.999736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268659b-ff0c-4167-b1b5-f85384c1362f None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:47.000921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268659b-ff0c-4167-b1b5-f85384c1362f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:47.000921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268659b-ff0c-4167-b1b5-f85384c1362f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:47.007345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268659b-ff0c-4167-b1b5-f85384c1362f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:47.038590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 293/597] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Aug 22 17:05:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:47.045062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268659b-ff0c-4167-b1b5-f85384c1362f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:47.101499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e950abe-7826-4d3a-b24d-bd32a87e1a10 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:47.180653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c244a-4b73-4716-8724-ceaa8dc55b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:47.181284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c244a-4b73-4716-8724-ceaa8dc55b2e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:47.181732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c244a-4b73-4716-8724-ceaa8dc55b2e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:47.182157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c244a-4b73-4716-8724-ceaa8dc55b2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:47.184935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:47.204097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c244a-4b73-4716-8724-ceaa8dc55b2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:47.208836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c244a-4b73-4716-8724-ceaa8dc55b2e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ad8c244a-4b73-4716-8724-ceaa8dc55b2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:47.241709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:47.243514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:47.245237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:47.245237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:47.245237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:47.245237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 305/598] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:47.252435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c244a-4b73-4716-8724-ceaa8dc55b2e None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:47.270218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268659b-ff0c-4167-b1b5-f85384c1362f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:47.329953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7fa20-eef5-4394-85c7-8139780c4e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:47.332292 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7fa20-eef5-4394-85c7-8139780c4e2a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:47.333092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7fa20-eef5-4394-85c7-8139780c4e2a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:47.333896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7fa20-eef5-4394-85c7-8139780c4e2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:47.362659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7fa20-eef5-4394-85c7-8139780c4e2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:47.365649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7fa20-eef5-4394-85c7-8139780c4e2a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:47.403873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-d5e7fa20-eef5-4394-85c7-8139780c4e2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:47.414516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:47.416401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:47.418133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:47.418133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:47.418133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:47.418133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 294/599] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:47.422512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7fa20-eef5-4394-85c7-8139780c4e2a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:47.445937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c244a-4b73-4716-8724-ceaa8dc55b2e None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:47.534243 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b866-3c80-4e49-a5e2-33f615cf892f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:47.534789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b866-3c80-4e49-a5e2-33f615cf892f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:47.535064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b866-3c80-4e49-a5e2-33f615cf892f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:47.535338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b866-3c80-4e49-a5e2-33f615cf892f None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:47.535968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1b866-3c80-4e49-a5e2-33f615cf892f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:47.536264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1b866-3c80-4e49-a5e2-33f615cf892f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:47.537636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1b866-3c80-4e49-a5e2-33f615cf892f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:47.570071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 306/600] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:47.575206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b866-3c80-4e49-a5e2-33f615cf892f None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:47.622930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7fa20-eef5-4394-85c7-8139780c4e2a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:47.671279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:47.702077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95347cdc-db57-4fac-a0f1-a0589235aec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:47.702715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95347cdc-db57-4fac-a0f1-a0589235aec4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:47.705002 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95347cdc-db57-4fac-a0f1-a0589235aec4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:47.705286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95347cdc-db57-4fac-a0f1-a0589235aec4 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:47.705950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95347cdc-db57-4fac-a0f1-a0589235aec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:47.706252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95347cdc-db57-4fac-a0f1-a0589235aec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:47.707674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95347cdc-db57-4fac-a0f1-a0589235aec4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:47.737932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 295/601] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:47.743965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95347cdc-db57-4fac-a0f1-a0589235aec4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:47.788072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b866-3c80-4e49-a5e2-33f615cf892f None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:47.836697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d22ed-ff41-4cf0-af41-87b703dc0635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:47.837346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d22ed-ff41-4cf0-af41-87b703dc0635 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:47.837632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d22ed-ff41-4cf0-af41-87b703dc0635 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:47.837895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d22ed-ff41-4cf0-af41-87b703dc0635 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:47.851775 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d22ed-ff41-4cf0-af41-87b703dc0635 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:47.853344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d22ed-ff41-4cf0-af41-87b703dc0635 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-279d22ed-ff41-4cf0-af41-87b703dc0635 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:47.866536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:47.867505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:47.868395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:47.868395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:47.868395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:47.868395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:47.868395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:47.868395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:47.868395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:47.868395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:47.872592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 307/602] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:47.873951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df48db0-3b66-4dfd-a41b-436312639ff2 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:47.874240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df48db0-3b66-4dfd-a41b-436312639ff2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:47.874538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df48db0-3b66-4dfd-a41b-436312639ff2 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:47.876959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 308/603] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:47.877667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df48db0-3b66-4dfd-a41b-436312639ff2 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:47.891053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:47.914518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95347cdc-db57-4fac-a0f1-a0589235aec4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:47.953413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002208f-b48c-4ec4-93f1-83934576e0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:47.954050 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002208f-b48c-4ec4-93f1-83934576e0c4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:47.955319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002208f-b48c-4ec4-93f1-83934576e0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:47.955823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002208f-b48c-4ec4-93f1-83934576e0c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:47.967324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002208f-b48c-4ec4-93f1-83934576e0c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:47.970406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002208f-b48c-4ec4-93f1-83934576e0c4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-f002208f-b48c-4ec4-93f1-83934576e0c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:47.989713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:47.991485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:47.994013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:47.994013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:47.994013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:47.994013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:47.994013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:47.994013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:47.994013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:47.994013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:47.997850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 296/604] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:47.999034 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8bffc-9c7c-4e35-9d39-d1b73a1cba4d None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:47.999316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8bffc-9c7c-4e35-9d39-d1b73a1cba4d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:47.999592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8bffc-9c7c-4e35-9d39-d1b73a1cba4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:48.018308 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-36f8bffc-9c7c-4e35-9d39-d1b73a1cba4d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:48.019558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-36f8bffc-9c7c-4e35-9d39-d1b73a1cba4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:48.030722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df48db0-3b66-4dfd-a41b-436312639ff2 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:48.053149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:48.086157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e390c7-8ae7-4062-bc24-90d2949bf631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:48.087966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e390c7-8ae7-4062-bc24-90d2949bf631 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:48.088752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e390c7-8ae7-4062-bc24-90d2949bf631 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:48.089224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e390c7-8ae7-4062-bc24-90d2949bf631 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:48.090831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e390c7-8ae7-4062-bc24-90d2949bf631 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:48.091087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e390c7-8ae7-4062-bc24-90d2949bf631 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:48.092546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e390c7-8ae7-4062-bc24-90d2949bf631 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:48.103807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 297/605] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:48.104680 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8bffc-9c7c-4e35-9d39-d1b73a1cba4d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:48.116303 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 309/606] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:48.118918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e390c7-8ae7-4062-bc24-90d2949bf631 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:48.223083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:48.272278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8bffc-9c7c-4e35-9d39-d1b73a1cba4d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:48.290268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e390c7-8ae7-4062-bc24-90d2949bf631 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:48.314965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111af06f-0563-4aea-8465-77654ce6aed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:48.315962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111af06f-0563-4aea-8465-77654ce6aed6 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:48.316475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111af06f-0563-4aea-8465-77654ce6aed6 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:48.317736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111af06f-0563-4aea-8465-77654ce6aed6 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:48.318678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111af06f-0563-4aea-8465-77654ce6aed6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:48.319971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111af06f-0563-4aea-8465-77654ce6aed6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:48.321857 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111af06f-0563-4aea-8465-77654ce6aed6 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:48.349362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 298/607] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:48.354995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111af06f-0563-4aea-8465-77654ce6aed6 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:48.361407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2644a-2104-4735-8f6a-61cf739279f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:48.362193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2644a-2104-4735-8f6a-61cf739279f0 None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:48.362474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2644a-2104-4735-8f6a-61cf739279f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:48.362790 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2644a-2104-4735-8f6a-61cf739279f0 None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/68088e59a2c64ff2948e0549f1fc7cb9/roles/14a34ac79fe64a978eca3e5f9089e72f` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:48.385533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2644a-2104-4735-8f6a-61cf739279f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=68088e59a2c64ff2948e0549f1fc7cb9, role_id=14a34ac79fe64a978eca3e5f9089e72f)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:48.394513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2644a-2104-4735-8f6a-61cf739279f0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:48.397288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:48.479162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111af06f-0563-4aea-8465-77654ce6aed6 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:48.517481 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7e3df-7e30-4aea-9d23-2ddca24c6f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:48.518394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7e3df-7e30-4aea-9d23-2ddca24c6f53 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:48.518786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7e3df-7e30-4aea-9d23-2ddca24c6f53 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:48.519154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7e3df-7e30-4aea-9d23-2ddca24c6f53 None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/c29a91e099ba4cd5886fd910d978fb4d/roles/14a34ac79fe64a978eca3e5f9089e72f` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:48.538316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 310/608] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:05:48 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/68088e59a2c64ff2948e0549f1fc7cb9/roles/14a34ac79fe64a978eca3e5f9089e72f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:05:48.539538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2644a-2104-4735-8f6a-61cf739279f0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:48.539769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7e3df-7e30-4aea-9d23-2ddca24c6f53 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=c29a91e099ba4cd5886fd910d978fb4d, role_id=14a34ac79fe64a978eca3e5f9089e72f)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:48.541652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7e3df-7e30-4aea-9d23-2ddca24c6f53 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:48.645518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 299/609] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:05:48 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/c29a91e099ba4cd5886fd910d978fb4d/roles/14a34ac79fe64a978eca3e5f9089e72f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:05:48.690326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2644a-2104-4735-8f6a-61cf739279f0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:48.710896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:48.737875 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612856c-160e-45a1-b78b-f1ed15c4bde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:48.738768 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612856c-160e-45a1-b78b-f1ed15c4bde0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:48.739146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612856c-160e-45a1-b78b-f1ed15c4bde0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:48.739514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612856c-160e-45a1-b78b-f1ed15c4bde0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:48.757335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612856c-160e-45a1-b78b-f1ed15c4bde0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-8612856c-160e-45a1-b78b-f1ed15c4bde0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:48.762913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:48.764126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:48.765725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:48.770528 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 311/610] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:48.774095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7e3df-7e30-4aea-9d23-2ddca24c6f53 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:48.870750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:48.898063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7e3df-7e30-4aea-9d23-2ddca24c6f53 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:48.937614 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7803b-a2d3-4c99-86e3-ec97941930a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:48.939035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7803b-a2d3-4c99-86e3-ec97941930a1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:48.939559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7803b-a2d3-4c99-86e3-ec97941930a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:48.939928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7803b-a2d3-4c99-86e3-ec97941930a1 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:48.940697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7803b-a2d3-4c99-86e3-ec97941930a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:48.941115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7803b-a2d3-4c99-86e3-ec97941930a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:48.942619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7803b-a2d3-4c99-86e3-ec97941930a1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:48.951938 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 300/611] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:48.958371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612856c-160e-45a1-b78b-f1ed15c4bde0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:49.066121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612856c-160e-45a1-b78b-f1ed15c4bde0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:49.102941 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764bbdf-9c10-41a1-9965-f234bce866a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:49.103734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764bbdf-9c10-41a1-9965-f234bce866a3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:49.104104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764bbdf-9c10-41a1-9965-f234bce866a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:49.104492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764bbdf-9c10-41a1-9965-f234bce866a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:49.115482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764bbdf-9c10-41a1-9965-f234bce866a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:49.117101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764bbdf-9c10-41a1-9965-f234bce866a3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-8764bbdf-9c10-41a1-9965-f234bce866a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:49.132811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:49.134006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:49.135112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:49.135112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:49.135112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:49.136408 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 312/612] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:49.142693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7803b-a2d3-4c99-86e3-ec97941930a1 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:49.220389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292c25f-a871-4489-9ba1-2792e3e014a0 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:49.220909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292c25f-a871-4489-9ba1-2792e3e014a0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:49.221509 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292c25f-a871-4489-9ba1-2792e3e014a0 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:49.223001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 313/613] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:05:49.228393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2518f0-9835-4274-9db5-af7ef16f52c6 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:49.228673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2518f0-9835-4274-9db5-af7ef16f52c6 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:49.228944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2518f0-9835-4274-9db5-af7ef16f52c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:49.254858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1b2518f0-9835-4274-9db5-af7ef16f52c6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:49.256116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1b2518f0-9835-4274-9db5-af7ef16f52c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:49.283519 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:49.306454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7803b-a2d3-4c99-86e3-ec97941930a1 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:49.370385 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 314/614] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:49.374810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599acc6-1efc-4cf8-90b6-8bc2dd176280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:49.375473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599acc6-1efc-4cf8-90b6-8bc2dd176280 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:49.375755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599acc6-1efc-4cf8-90b6-8bc2dd176280 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:49.376783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599acc6-1efc-4cf8-90b6-8bc2dd176280 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:49.377448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599acc6-1efc-4cf8-90b6-8bc2dd176280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:49.377750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599acc6-1efc-4cf8-90b6-8bc2dd176280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:49.379206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599acc6-1efc-4cf8-90b6-8bc2dd176280 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:49.387269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:49.393663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 301/615] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:49.399385 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2518f0-9835-4274-9db5-af7ef16f52c6 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:49.531182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2518f0-9835-4274-9db5-af7ef16f52c6 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:49.584980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c9376-c2d3-4842-a2d2-30f03d2b2ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:49.586412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c9376-c2d3-4842-a2d2-30f03d2b2ee5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:49.586913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c9376-c2d3-4842-a2d2-30f03d2b2ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:49.587520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c9376-c2d3-4842-a2d2-30f03d2b2ee5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:49.602987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c9376-c2d3-4842-a2d2-30f03d2b2ee5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:49.604720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c9376-c2d3-4842-a2d2-30f03d2b2ee5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-546c9376-c2d3-4842-a2d2-30f03d2b2ee5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:49.622679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:49.623595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:49.624534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 315/616] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:05:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:49.631559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599acc6-1efc-4cf8-90b6-8bc2dd176280 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:49.669026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c9376-c2d3-4842-a2d2-30f03d2b2ee5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:49.706615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:49.762220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599acc6-1efc-4cf8-90b6-8bc2dd176280 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:49.802410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f5d00-d995-4cc4-8d99-fb3bd9baf0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:49.803285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f5d00-d995-4cc4-8d99-fb3bd9baf0be None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:49.803707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f5d00-d995-4cc4-8d99-fb3bd9baf0be None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:49.804116 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f5d00-d995-4cc4-8d99-fb3bd9baf0be None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:49.805101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f5d00-d995-4cc4-8d99-fb3bd9baf0be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:49.805593 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f5d00-d995-4cc4-8d99-fb3bd9baf0be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:49.807197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f5d00-d995-4cc4-8d99-fb3bd9baf0be None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:49.808094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c9376-c2d3-4842-a2d2-30f03d2b2ee5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:49.831863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 302/617] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 22 17:05:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:49.833225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe3722-91fc-49d2-8c94-9b6252314c0b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:49.833509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe3722-91fc-49d2-8c94-9b6252314c0b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:49.833778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe3722-91fc-49d2-8c94-9b6252314c0b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:49.839683 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 303/618] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:49.841190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe3722-91fc-49d2-8c94-9b6252314c0b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:49.848040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3679bd8-c44f-4762-8099-56bdc425f089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:49.848689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3679bd8-c44f-4762-8099-56bdc425f089 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:49.848968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3679bd8-c44f-4762-8099-56bdc425f089 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:49.849241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3679bd8-c44f-4762-8099-56bdc425f089 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:49.863315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3679bd8-c44f-4762-8099-56bdc425f089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-a3679bd8-c44f-4762-8099-56bdc425f089 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:49.869718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:49.870835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:49.871672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 316/619] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:49.873203 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc845d79-8b74-4ef0-8a19-ff00b0494e85 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:49.874271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc845d79-8b74-4ef0-8a19-ff00b0494e85 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:49.874271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc845d79-8b74-4ef0-8a19-ff00b0494e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:49.896731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-dc845d79-8b74-4ef0-8a19-ff00b0494e85 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:49.897917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dc845d79-8b74-4ef0-8a19-ff00b0494e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:49.906849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:49.965812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe3722-91fc-49d2-8c94-9b6252314c0b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:49.979048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 317/620] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:49.987487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc845d79-8b74-4ef0-8a19-ff00b0494e85 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:50.004707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:50.005239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced66084-ff18-40e8-9be4-52ecdc1a22e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:50.006285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced66084-ff18-40e8-9be4-52ecdc1a22e4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:50.006585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced66084-ff18-40e8-9be4-52ecdc1a22e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:50.006860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced66084-ff18-40e8-9be4-52ecdc1a22e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:50.019835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced66084-ff18-40e8-9be4-52ecdc1a22e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:50.021309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced66084-ff18-40e8-9be4-52ecdc1a22e4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-ced66084-ff18-40e8-9be4-52ecdc1a22e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:50.039579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:50.040589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:50.041489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:50.041489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:50.041489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:50.041489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 304/621] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:50.047998 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced66084-ff18-40e8-9be4-52ecdc1a22e4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:50.166278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced66084-ff18-40e8-9be4-52ecdc1a22e4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:50.177641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:50.208515 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc845d79-8b74-4ef0-8a19-ff00b0494e85 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:50.208841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd706b-d52c-42a8-b24d-69e9dbff462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:50.209258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd706b-d52c-42a8-b24d-69e9dbff462c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:50.209539 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd706b-d52c-42a8-b24d-69e9dbff462c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:50.209806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd706b-d52c-42a8-b24d-69e9dbff462c None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:50.210784 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd706b-d52c-42a8-b24d-69e9dbff462c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:50.212066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd706b-d52c-42a8-b24d-69e9dbff462c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:50.213651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd706b-d52c-42a8-b24d-69e9dbff462c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:50.227649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 305/622] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:50.235377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd706b-d52c-42a8-b24d-69e9dbff462c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:50.277004 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d6651-768d-46c6-8441-ea3290f6372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:50.277618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d6651-768d-46c6-8441-ea3290f6372d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:50.277893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d6651-768d-46c6-8441-ea3290f6372d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:50.278161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d6651-768d-46c6-8441-ea3290f6372d None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:50.279110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d6651-768d-46c6-8441-ea3290f6372d None admin] RBAC: Adding query filter params (name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:50.279408 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d6651-768d-46c6-8441-ea3290f6372d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:50.280772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d6651-768d-46c6-8441-ea3290f6372d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:50.297112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 318/623] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:50.303001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d6651-768d-46c6-8441-ea3290f6372d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:50.390019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd706b-d52c-42a8-b24d-69e9dbff462c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:50.433214 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20348caf-11e8-4967-a67a-6ec9ef4d748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:50.433762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20348caf-11e8-4967-a67a-6ec9ef4d748f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:50.434034 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20348caf-11e8-4967-a67a-6ec9ef4d748f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:50.434301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20348caf-11e8-4967-a67a-6ec9ef4d748f None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:50.442519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20348caf-11e8-4967-a67a-6ec9ef4d748f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:50.444787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20348caf-11e8-4967-a67a-6ec9ef4d748f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-20348caf-11e8-4967-a67a-6ec9ef4d748f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:50.458642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:50.459619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:50.460502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:50.460502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:50.460502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:50.460502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:50.460502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:50.460502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:50.460502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:50.460502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:50.464024 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 306/624] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:50.464596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:50.465677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20348caf-11e8-4967-a67a-6ec9ef4d748f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:50.499241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:50.502141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d6651-768d-46c6-8441-ea3290f6372d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:50.538931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de5644-5409-4910-8bbf-f220c79d84fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:50.539623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de5644-5409-4910-8bbf-f220c79d84fa None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:50.540022 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de5644-5409-4910-8bbf-f220c79d84fa None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:50.540206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de5644-5409-4910-8bbf-f220c79d84fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:50.552758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de5644-5409-4910-8bbf-f220c79d84fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:50.554247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de5644-5409-4910-8bbf-f220c79d84fa None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-c9de5644-5409-4910-8bbf-f220c79d84fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:50.569934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:50.571433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:50.572695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:50.572695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:50.572695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:50.576188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 319/625] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:50.576947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de5644-5409-4910-8bbf-f220c79d84fa None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:50.586903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20348caf-11e8-4967-a67a-6ec9ef4d748f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:50.634449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72a884-ae6e-4ddf-8acc-b1cefa730685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:50.635420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72a884-ae6e-4ddf-8acc-b1cefa730685 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:50.635919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72a884-ae6e-4ddf-8acc-b1cefa730685 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:50.637003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72a884-ae6e-4ddf-8acc-b1cefa730685 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:50.653434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72a884-ae6e-4ddf-8acc-b1cefa730685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-0c72a884-ae6e-4ddf-8acc-b1cefa730685 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:50.661774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:50.662765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:50.663631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:50.664263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 307/626] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:50.669917 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72a884-ae6e-4ddf-8acc-b1cefa730685 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:50.711087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de5644-5409-4910-8bbf-f220c79d84fa None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:50.740960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:50.754370 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673e522-ee31-483e-8dcf-201c0b9ca714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:50.755164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673e522-ee31-483e-8dcf-201c0b9ca714 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:50.755679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673e522-ee31-483e-8dcf-201c0b9ca714 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:50.755980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673e522-ee31-483e-8dcf-201c0b9ca714 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:50.756834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673e522-ee31-483e-8dcf-201c0b9ca714 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:50.757229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673e522-ee31-483e-8dcf-201c0b9ca714 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:50.758925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673e522-ee31-483e-8dcf-201c0b9ca714 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:50.778244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 320/627] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:50.779430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673e522-ee31-483e-8dcf-201c0b9ca714 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:50.787686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:50.802979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72a884-ae6e-4ddf-8acc-b1cefa730685 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:50.848123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f638378-5f28-447a-b858-0f7d2fa39602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:50.848709 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f638378-5f28-447a-b858-0f7d2fa39602 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:50.849043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f638378-5f28-447a-b858-0f7d2fa39602 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:50.849320 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f638378-5f28-447a-b858-0f7d2fa39602 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:50.849972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f638378-5f28-447a-b858-0f7d2fa39602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:50.850329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f638378-5f28-447a-b858-0f7d2fa39602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:50.851777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f638378-5f28-447a-b858-0f7d2fa39602 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:50.867990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 308/628] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:50.868859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f638378-5f28-447a-b858-0f7d2fa39602 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:50.897584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673e522-ee31-483e-8dcf-201c0b9ca714 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:50.940512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165e0f7-f9d4-47fc-abdc-71639de3e083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:50.941119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165e0f7-f9d4-47fc-abdc-71639de3e083 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:50.941395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165e0f7-f9d4-47fc-abdc-71639de3e083 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:50.941663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165e0f7-f9d4-47fc-abdc-71639de3e083 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:50.942327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165e0f7-f9d4-47fc-abdc-71639de3e083 None admin] RBAC: Adding query filter params (name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:50.942809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165e0f7-f9d4-47fc-abdc-71639de3e083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:50.944684 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165e0f7-f9d4-47fc-abdc-71639de3e083 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:50.956691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 321/629] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:50.958471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226766a-ec06-4b6d-ab31-293bb527829b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:50.958842 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226766a-ec06-4b6d-ab31-293bb527829b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:50.959182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226766a-ec06-4b6d-ab31-293bb527829b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:50.962306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 322/630] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:50.962306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226766a-ec06-4b6d-ab31-293bb527829b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:50.990705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f638378-5f28-447a-b858-0f7d2fa39602 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:51.015782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:51.037340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533ce94-ca30-45cf-a87b-ce2cacfd2e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:51.038209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533ce94-ca30-45cf-a87b-ce2cacfd2e0f None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:51.038536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533ce94-ca30-45cf-a87b-ce2cacfd2e0f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:51.038820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533ce94-ca30-45cf-a87b-ce2cacfd2e0f None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/38462b1165514ed7b948cba710782d4b/roles/14a34ac79fe64a978eca3e5f9089e72f` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:51.057642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533ce94-ca30-45cf-a87b-ce2cacfd2e0f None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=38462b1165514ed7b948cba710782d4b, role_id=14a34ac79fe64a978eca3e5f9089e72f)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:51.059254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533ce94-ca30-45cf-a87b-ce2cacfd2e0f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:51.078570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226766a-ec06-4b6d-ab31-293bb527829b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:51.113171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe9e1a-900d-498f-9074-d3b225688d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:51.116646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe9e1a-900d-498f-9074-d3b225688d34 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:51.116646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe9e1a-900d-498f-9074-d3b225688d34 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:51.116646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe9e1a-900d-498f-9074-d3b225688d34 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:51.123872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe9e1a-900d-498f-9074-d3b225688d34 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:51.125325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe9e1a-900d-498f-9074-d3b225688d34 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-aafe9e1a-900d-498f-9074-d3b225688d34 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:51.138309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:51.139352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 323/631] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:51.140300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe9e1a-900d-498f-9074-d3b225688d34 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:51.147212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:51.152549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 309/632] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:05:50 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/38462b1165514ed7b948cba710782d4b/roles/14a34ac79fe64a978eca3e5f9089e72f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:05:51.157121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c74f6-fbb5-4ab8-a751-427d5355aaf6 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:51.157121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c74f6-fbb5-4ab8-a751-427d5355aaf6 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:51.157121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c74f6-fbb5-4ab8-a751-427d5355aaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:51.179184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-4a2c74f6-fbb5-4ab8-a751-427d5355aaf6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:51.180420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4a2c74f6-fbb5-4ab8-a751-427d5355aaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:51.273127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 310/633] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:51.274856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efd9dd-6bba-48bd-959f-c6e0e6f32292 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:51.275207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efd9dd-6bba-48bd-959f-c6e0e6f32292 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:51.275553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efd9dd-6bba-48bd-959f-c6e0e6f32292 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:51.277226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 311/634] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:51.282330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efd9dd-6bba-48bd-959f-c6e0e6f32292 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:51.287943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe9e1a-900d-498f-9074-d3b225688d34 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:51.291248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:51.331261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b1284-b1ab-4424-8bb5-f54cd7928f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:51.331909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1284-b1ab-4424-8bb5-f54cd7928f86 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:51.332248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1284-b1ab-4424-8bb5-f54cd7928f86 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:51.332521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1284-b1ab-4424-8bb5-f54cd7928f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:51.344932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b1284-b1ab-4424-8bb5-f54cd7928f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:51.352676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b1284-b1ab-4424-8bb5-f54cd7928f86 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-971b1284-b1ab-4424-8bb5-f54cd7928f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:51.368399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:51.369488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:51.371091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:51.371091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:51.371091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:51.371091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 324/635] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:51.372772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ead5-b93c-4c01-af11-9b13f3bf6370 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:51.372772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ead5-b93c-4c01-af11-9b13f3bf6370 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:51.372772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ead5-b93c-4c01-af11-9b13f3bf6370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:51.392986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-7ea8ead5-b93c-4c01-af11-9b13f3bf6370 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:51.394200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7ea8ead5-b93c-4c01-af11-9b13f3bf6370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:51.413407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efd9dd-6bba-48bd-959f-c6e0e6f32292 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:51.449567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:51.460666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b292c-c301-4829-9132-48a8804ee239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:51.461629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b292c-c301-4829-9132-48a8804ee239 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:51.461909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b292c-c301-4829-9132-48a8804ee239 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:51.462180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b292c-c301-4829-9132-48a8804ee239 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:51.463500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b292c-c301-4829-9132-48a8804ee239 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:51.463988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b292c-c301-4829-9132-48a8804ee239 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:51.466675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b292c-c301-4829-9132-48a8804ee239 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:51.489133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 312/636] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 22 17:05:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:51.490525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b292c-c301-4829-9132-48a8804ee239 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:51.510717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 325/637] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:51.515205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8ead5-b93c-4c01-af11-9b13f3bf6370 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:51.687074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:51.699952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b292c-c301-4829-9132-48a8804ee239 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:51.705804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8ead5-b93c-4c01-af11-9b13f3bf6370 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:51.750392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bed25-d444-476d-bb55-d09609a8e9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:51.751565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bed25-d444-476d-bb55-d09609a8e9db None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:51.752507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bed25-d444-476d-bb55-d09609a8e9db None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:51.753261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bed25-d444-476d-bb55-d09609a8e9db None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:51.754364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bed25-d444-476d-bb55-d09609a8e9db None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:51.755662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bed25-d444-476d-bb55-d09609a8e9db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:51.757919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bed25-d444-476d-bb55-d09609a8e9db None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:51.760448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24f19d-8bc2-4ab5-91ec-0c7c32b82475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:51.761171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24f19d-8bc2-4ab5-91ec-0c7c32b82475 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:51.761456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24f19d-8bc2-4ab5-91ec-0c7c32b82475 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:51.761730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24f19d-8bc2-4ab5-91ec-0c7c32b82475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:51.775025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 313/638] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:51.776156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bed25-d444-476d-bb55-d09609a8e9db None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:51.852342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:51.882565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24f19d-8bc2-4ab5-91ec-0c7c32b82475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:51.884500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24f19d-8bc2-4ab5-91ec-0c7c32b82475 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:51.922229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bed25-d444-476d-bb55-d09609a8e9db None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:51.970163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a84c0-7675-4301-886c-8f0c0081aeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:51.970825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a84c0-7675-4301-886c-8f0c0081aeed None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:51.971128 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a84c0-7675-4301-886c-8f0c0081aeed None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:51.971373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a84c0-7675-4301-886c-8f0c0081aeed None admin] PATH_INFO: `/v3/roles/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:51.982522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a84c0-7675-4301-886c-8f0c0081aeed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-025a84c0-7675-4301-886c-8f0c0081aeed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:51.988257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:51.990026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 314/639] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:51.991386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a84c0-7675-4301-886c-8f0c0081aeed None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.003390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:52.024467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 326/640] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:52.025163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24f19d-8bc2-4ab5-91ec-0c7c32b82475 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.114353 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a84c0-7675-4301-886c-8f0c0081aeed None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:52.121647 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:52.145989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24f19d-8bc2-4ab5-91ec-0c7c32b82475 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:52.150760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c88745-1b89-4a57-b9ec-99751b041ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:52.151512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c88745-1b89-4a57-b9ec-99751b041ce9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:52.151888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c88745-1b89-4a57-b9ec-99751b041ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:52.152257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c88745-1b89-4a57-b9ec-99751b041ce9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:52.160383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c88745-1b89-4a57-b9ec-99751b041ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:52.161977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c88745-1b89-4a57-b9ec-99751b041ce9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-43c88745-1b89-4a57-b9ec-99751b041ce9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:52.175404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:52.176576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:52.177735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:52.177735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:52.177735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:52.177735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:52.177735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:52.177735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:52.177735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:52.177735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:52.177735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:05:52.177735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:52.179610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 315/641] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 22 17:05:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:52.180829 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c88745-1b89-4a57-b9ec-99751b041ce9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.185589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858414a2-07a7-4a1f-870e-c105485fd896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:52.186229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858414a2-07a7-4a1f-870e-c105485fd896 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:52.186521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858414a2-07a7-4a1f-870e-c105485fd896 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:52.186794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858414a2-07a7-4a1f-870e-c105485fd896 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:52.198320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858414a2-07a7-4a1f-870e-c105485fd896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-858414a2-07a7-4a1f-870e-c105485fd896 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:52.203521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:52.205300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:52.206367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:52.210123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 327/642] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:52.211045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858414a2-07a7-4a1f-870e-c105485fd896 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.265186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:52.290550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c88745-1b89-4a57-b9ec-99751b041ce9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:52.327230 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858414a2-07a7-4a1f-870e-c105485fd896 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:52.328365 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abcef5-83ba-4719-a654-6795909de467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:52.329674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abcef5-83ba-4719-a654-6795909de467 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:52.329810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abcef5-83ba-4719-a654-6795909de467 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:52.330068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abcef5-83ba-4719-a654-6795909de467 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:52.330779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abcef5-83ba-4719-a654-6795909de467 None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:52.331080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abcef5-83ba-4719-a654-6795909de467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:52.332468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abcef5-83ba-4719-a654-6795909de467 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:52.343547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 316/643] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:52.345171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abcef5-83ba-4719-a654-6795909de467 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.373989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094603c8-399b-435d-843e-7194be791da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:52.374669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094603c8-399b-435d-843e-7194be791da2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:52.374948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094603c8-399b-435d-843e-7194be791da2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:52.375226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094603c8-399b-435d-843e-7194be791da2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:52.379024 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:52.386514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094603c8-399b-435d-843e-7194be791da2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:52.388011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094603c8-399b-435d-843e-7194be791da2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-094603c8-399b-435d-843e-7194be791da2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:52.404229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:52.405202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:52.406262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:52.406262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:52.406262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:52.410598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 328/644] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:52.411394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094603c8-399b-435d-843e-7194be791da2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.459910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abcef5-83ba-4719-a654-6795909de467 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:52.499224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7529b0c-41ae-42ed-a67b-11c1f2e02a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:52.499840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7529b0c-41ae-42ed-a67b-11c1f2e02a66 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:52.500114 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7529b0c-41ae-42ed-a67b-11c1f2e02a66 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:52.500383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7529b0c-41ae-42ed-a67b-11c1f2e02a66 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:52.501700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7529b0c-41ae-42ed-a67b-11c1f2e02a66 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:52.502055 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7529b0c-41ae-42ed-a67b-11c1f2e02a66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:52.503773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7529b0c-41ae-42ed-a67b-11c1f2e02a66 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:52.528872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 317/645] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 22 17:05:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:52.528872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7529b0c-41ae-42ed-a67b-11c1f2e02a66 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.530033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:52.535332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094603c8-399b-435d-843e-7194be791da2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:52.572866 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9297c-4c2d-409e-a6af-a83fa7f576c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:52.574021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9297c-4c2d-409e-a6af-a83fa7f576c2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:52.574559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9297c-4c2d-409e-a6af-a83fa7f576c2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:52.575016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9297c-4c2d-409e-a6af-a83fa7f576c2 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:52.575988 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9297c-4c2d-409e-a6af-a83fa7f576c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:52.576635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9297c-4c2d-409e-a6af-a83fa7f576c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:52.578086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9297c-4c2d-409e-a6af-a83fa7f576c2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:52.594267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 329/646] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:52.594267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9297c-4c2d-409e-a6af-a83fa7f576c2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.632281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:52.648500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7529b0c-41ae-42ed-a67b-11c1f2e02a66 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:52.689584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d51c6-a5b8-4992-9673-c8d3b486d2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:52.690372 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d51c6-a5b8-4992-9673-c8d3b486d2c8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:52.690773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d51c6-a5b8-4992-9673-c8d3b486d2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:52.691145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d51c6-a5b8-4992-9673-c8d3b486d2c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:52.702294 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d51c6-a5b8-4992-9673-c8d3b486d2c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:52.704730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d51c6-a5b8-4992-9673-c8d3b486d2c8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:52.711255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9297c-4c2d-409e-a6af-a83fa7f576c2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-8a9d51c6-a5b8-4992-9673-c8d3b486d2c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:52.721577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:52.722979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:52.723946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:52.723946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:52.723946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:52.723946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 318/647] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:52.724187 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d51c6-a5b8-4992-9673-c8d3b486d2c8 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.752475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aff81-72a5-4793-93ff-3050d3ee6c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:52.753113 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aff81-72a5-4793-93ff-3050d3ee6c32 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:52.753394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aff81-72a5-4793-93ff-3050d3ee6c32 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:52.753662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aff81-72a5-4793-93ff-3050d3ee6c32 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:52.754304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aff81-72a5-4793-93ff-3050d3ee6c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:52.754650 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aff81-72a5-4793-93ff-3050d3ee6c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:52.756039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aff81-72a5-4793-93ff-3050d3ee6c32 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:52.773516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 330/648] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:52.773516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aff81-72a5-4793-93ff-3050d3ee6c32 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.799076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:52.841055 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d51c6-a5b8-4992-9673-c8d3b486d2c8 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:52.876180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae1817-d3cd-4190-b7ac-f5403fa014e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:52.876950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae1817-d3cd-4190-b7ac-f5403fa014e8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:52.877247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae1817-d3cd-4190-b7ac-f5403fa014e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:52.877589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae1817-d3cd-4190-b7ac-f5403fa014e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:52.884615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aff81-72a5-4793-93ff-3050d3ee6c32 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:52.887879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:52.920979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0713d5d-bb7f-449f-a665-f9bab0ef2ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:52.921606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713d5d-bb7f-449f-a665-f9bab0ef2ce4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:52.921887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713d5d-bb7f-449f-a665-f9bab0ef2ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:52.922144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713d5d-bb7f-449f-a665-f9bab0ef2ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:52.932993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0713d5d-bb7f-449f-a665-f9bab0ef2ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:52.934451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0713d5d-bb7f-449f-a665-f9bab0ef2ce4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-c0713d5d-bb7f-449f-a665-f9bab0ef2ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:52.949522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:52.956290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:52.957391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:52.957391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:52.957391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:52.957391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 331/649] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:52.957391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0713d5d-bb7f-449f-a665-f9bab0ef2ce4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:52.961569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae1817-d3cd-4190-b7ac-f5403fa014e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:52.963170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae1817-d3cd-4190-b7ac-f5403fa014e8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.030294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:53.070776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 319/650] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:53.070971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0713d5d-bb7f-449f-a665-f9bab0ef2ce4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:53.071976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae1817-d3cd-4190-b7ac-f5403fa014e8 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:53.112265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfe066-36d3-418e-ab62-b36deb290877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:53.112884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfe066-36d3-418e-ab62-b36deb290877 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.113156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfe066-36d3-418e-ab62-b36deb290877 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.113427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfe066-36d3-418e-ab62-b36deb290877 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.114069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfe066-36d3-418e-ab62-b36deb290877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:53.114367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfe066-36d3-418e-ab62-b36deb290877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:53.115804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfe066-36d3-418e-ab62-b36deb290877 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.120221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:53.133670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 332/651] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:53.133670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfe066-36d3-418e-ab62-b36deb290877 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:53.196348 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae1817-d3cd-4190-b7ac-f5403fa014e8 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:53.233324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decee76-116f-4698-9afd-2039bdb63b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:53.233922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decee76-116f-4698-9afd-2039bdb63b84 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.234140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decee76-116f-4698-9afd-2039bdb63b84 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.234410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decee76-116f-4698-9afd-2039bdb63b84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.241448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decee76-116f-4698-9afd-2039bdb63b84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:53.241775 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfe066-36d3-418e-ab62-b36deb290877 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:53.242973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decee76-116f-4698-9afd-2039bdb63b84 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-7decee76-116f-4698-9afd-2039bdb63b84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:53.255549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:53.256657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:53.257820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:53.257820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:53.257820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:53.257820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:53.257820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:53.257820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:53.257820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:53.257820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:53.257820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 320/652] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:53.257820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decee76-116f-4698-9afd-2039bdb63b84 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:53.274894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:53.279706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf505d-fc89-4cc3-8922-4df39a0c8e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:53.280312 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf505d-fc89-4cc3-8922-4df39a0c8e7a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.280584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf505d-fc89-4cc3-8922-4df39a0c8e7a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.280850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf505d-fc89-4cc3-8922-4df39a0c8e7a None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.281484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf505d-fc89-4cc3-8922-4df39a0c8e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:53.281782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf505d-fc89-4cc3-8922-4df39a0c8e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:53.283478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf505d-fc89-4cc3-8922-4df39a0c8e7a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.294346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 333/653] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:53.295433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf505d-fc89-4cc3-8922-4df39a0c8e7a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:53.359984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:53.364077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decee76-116f-4698-9afd-2039bdb63b84 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:53.400670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98109c9b-6d76-4809-a15c-c96dc71ad059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:53.401293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98109c9b-6d76-4809-a15c-c96dc71ad059 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.401569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98109c9b-6d76-4809-a15c-c96dc71ad059 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.401841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98109c9b-6d76-4809-a15c-c96dc71ad059 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.404757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf505d-fc89-4cc3-8922-4df39a0c8e7a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:53.412463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98109c9b-6d76-4809-a15c-c96dc71ad059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:53.414040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98109c9b-6d76-4809-a15c-c96dc71ad059 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-98109c9b-6d76-4809-a15c-c96dc71ad059 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:53.429101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:53.430158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:53.431110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:53.431110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:53.431110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:53.431110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 321/654] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:53.431465 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aabebb-5eae-4131-9b3a-058842ec264f None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.431761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aabebb-5eae-4131-9b3a-058842ec264f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.432012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aabebb-5eae-4131-9b3a-058842ec264f None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.433428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 322/655] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:53.433981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aabebb-5eae-4131-9b3a-058842ec264f None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:53.442780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af8fec-5458-4516-9626-e9d2a2e1717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:53.443415 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af8fec-5458-4516-9626-e9d2a2e1717a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.443691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af8fec-5458-4516-9626-e9d2a2e1717a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.443957 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af8fec-5458-4516-9626-e9d2a2e1717a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.451771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af8fec-5458-4516-9626-e9d2a2e1717a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:53.453226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af8fec-5458-4516-9626-e9d2a2e1717a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-13af8fec-5458-4516-9626-e9d2a2e1717a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:53.464937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:53.465990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 334/656] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:53.467409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af8fec-5458-4516-9626-e9d2a2e1717a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:53.525006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:53.544359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aabebb-5eae-4131-9b3a-058842ec264f None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:53.578402 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af8fec-5458-4516-9626-e9d2a2e1717a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:53.581581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b1cf0-ee24-4918-9021-1ad31d9e246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:53.582186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b1cf0-ee24-4918-9021-1ad31d9e246b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.582467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b1cf0-ee24-4918-9021-1ad31d9e246b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.582779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b1cf0-ee24-4918-9021-1ad31d9e246b None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.583474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b1cf0-ee24-4918-9021-1ad31d9e246b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:53.583781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b1cf0-ee24-4918-9021-1ad31d9e246b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:53.585228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b1cf0-ee24-4918-9021-1ad31d9e246b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.599737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 323/657] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:53.600583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b1cf0-ee24-4918-9021-1ad31d9e246b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:53.614174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:53.615094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453dc2da-796b-476d-8745-a2e2cc29f57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:53.615730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dc2da-796b-476d-8745-a2e2cc29f57c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.615999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dc2da-796b-476d-8745-a2e2cc29f57c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.616268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dc2da-796b-476d-8745-a2e2cc29f57c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.627241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453dc2da-796b-476d-8745-a2e2cc29f57c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:53.628736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453dc2da-796b-476d-8745-a2e2cc29f57c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-453dc2da-796b-476d-8745-a2e2cc29f57c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:53.639774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:53.640899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:53.641897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 335/658] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:53.642992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10521ae-dc4f-48b8-b2ab-76778ba395fa None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.642992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10521ae-dc4f-48b8-b2ab-76778ba395fa None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.642992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10521ae-dc4f-48b8-b2ab-76778ba395fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.660238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d10521ae-dc4f-48b8-b2ab-76778ba395fa None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:53.661462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d10521ae-dc4f-48b8-b2ab-76778ba395fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:53.706723 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b1cf0-ee24-4918-9021-1ad31d9e246b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:53.729830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 336/659] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:53.730688 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10521ae-dc4f-48b8-b2ab-76778ba395fa None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:53.740934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fde64-e05c-429f-a9b7-b9b52071b603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:53.741805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fde64-e05c-429f-a9b7-b9b52071b603 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.741805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fde64-e05c-429f-a9b7-b9b52071b603 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.742080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fde64-e05c-429f-a9b7-b9b52071b603 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.742757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fde64-e05c-429f-a9b7-b9b52071b603 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:53.743061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fde64-e05c-429f-a9b7-b9b52071b603 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:53.744432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fde64-e05c-429f-a9b7-b9b52071b603 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.757273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 324/660] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:53.757937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fde64-e05c-429f-a9b7-b9b52071b603 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:53.784848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:53.833338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:53.846333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10521ae-dc4f-48b8-b2ab-76778ba395fa None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:53.867736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fde64-e05c-429f-a9b7-b9b52071b603 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:53.883575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcd022-8679-4d87-ab9c-59dec3266fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:53.884185 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcd022-8679-4d87-ab9c-59dec3266fe0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.884461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcd022-8679-4d87-ab9c-59dec3266fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.884732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcd022-8679-4d87-ab9c-59dec3266fe0 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.885456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcd022-8679-4d87-ab9c-59dec3266fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:53.885759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcd022-8679-4d87-ab9c-59dec3266fe0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:53.887228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcd022-8679-4d87-ab9c-59dec3266fe0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.904293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b366dd3-2cee-4bfd-aa65-c51c8a10dea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:53.909197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b366dd3-2cee-4bfd-aa65-c51c8a10dea9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:53.909359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 337/661] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:53.909464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b366dd3-2cee-4bfd-aa65-c51c8a10dea9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:53.909464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b366dd3-2cee-4bfd-aa65-c51c8a10dea9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:53.909627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcd022-8679-4d87-ab9c-59dec3266fe0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:53.909711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b366dd3-2cee-4bfd-aa65-c51c8a10dea9 None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=ea4c2870e2754f3188228fc441950df2) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:53.909711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b366dd3-2cee-4bfd-aa65-c51c8a10dea9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:53.909711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b366dd3-2cee-4bfd-aa65-c51c8a10dea9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:53.914900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 325/662] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/role_assignments?user.id=ea4c2870e2754f3188228fc441950df2&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:53.919433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b366dd3-2cee-4bfd-aa65-c51c8a10dea9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:54.030800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:54.036477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcd022-8679-4d87-ab9c-59dec3266fe0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:54.058407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b366dd3-2cee-4bfd-aa65-c51c8a10dea9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:54.079226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b884-6073-4e03-8f23-658fcaac6903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:54.080124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b884-6073-4e03-8f23-658fcaac6903 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:54.080232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b884-6073-4e03-8f23-658fcaac6903 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:54.080478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b884-6073-4e03-8f23-658fcaac6903 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:54.082182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5b884-6073-4e03-8f23-658fcaac6903 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:54.082182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5b884-6073-4e03-8f23-658fcaac6903 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:54.082967 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5b884-6073-4e03-8f23-658fcaac6903 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:54.099950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d509d6-2d3f-460e-a6f4-3a3322547433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:54.100728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d509d6-2d3f-460e-a6f4-3a3322547433 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:54.101104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d509d6-2d3f-460e-a6f4-3a3322547433 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:54.101480 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d509d6-2d3f-460e-a6f4-3a3322547433 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:54.103713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:54.106228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 338/663] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:54.106379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b884-6073-4e03-8f23-658fcaac6903 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:54.109096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d509d6-2d3f-460e-a6f4-3a3322547433 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:54.111246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d509d6-2d3f-460e-a6f4-3a3322547433 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-e3d509d6-2d3f-460e-a6f4-3a3322547433 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:54.125761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:54.126884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:54.128004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:54.128004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:54.128004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:54.128004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:54.128004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:54.128004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:54.128004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:54.128004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:54.131255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 326/664] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:54.131582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d509d6-2d3f-460e-a6f4-3a3322547433 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:54.219209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b884-6073-4e03-8f23-658fcaac6903 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:54.249060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d509d6-2d3f-460e-a6f4-3a3322547433 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:54.261863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e8bfd-db80-4fe6-8d33-484c9ec10e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:54.262456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e8bfd-db80-4fe6-8d33-484c9ec10e6b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:54.262769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e8bfd-db80-4fe6-8d33-484c9ec10e6b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:54.263038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e8bfd-db80-4fe6-8d33-484c9ec10e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:54.285793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9a0b7-7f7d-4c85-95f8-9696be86694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:54.286689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9a0b7-7f7d-4c85-95f8-9696be86694b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:54.286994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9a0b7-7f7d-4c85-95f8-9696be86694b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:54.287354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9a0b7-7f7d-4c85-95f8-9696be86694b None admin] PATH_INFO: `/v3/roles/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:54.299528 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9a0b7-7f7d-4c85-95f8-9696be86694b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:54.300097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-a6b9a0b7-7f7d-4c85-95f8-9696be86694b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:54.305363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:54.306560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:05:54.307583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:54.311293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 327/665] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:54.312186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9a0b7-7f7d-4c85-95f8-9696be86694b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:54.348808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e8bfd-db80-4fe6-8d33-484c9ec10e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:54.350539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e8bfd-db80-4fe6-8d33-484c9ec10e6b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:54.381311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:54.431437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9a0b7-7f7d-4c85-95f8-9696be86694b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:54.466818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 339/666] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:54.467870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e8bfd-db80-4fe6-8d33-484c9ec10e6b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:54.471606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10863e60-8b5e-4668-a4db-df03ed551314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:54.472438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10863e60-8b5e-4668-a4db-df03ed551314 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:54.472714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10863e60-8b5e-4668-a4db-df03ed551314 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:54.472983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10863e60-8b5e-4668-a4db-df03ed551314 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:54.535302 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:54.551044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10863e60-8b5e-4668-a4db-df03ed551314 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:54.552540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10863e60-8b5e-4668-a4db-df03ed551314 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:54.589291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e8bfd-db80-4fe6-8d33-484c9ec10e6b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:54.609688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:54.633691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ebd53-eb7a-4877-b567-450f72419896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:54.634309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ebd53-eb7a-4877-b567-450f72419896 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:54.634622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ebd53-eb7a-4877-b567-450f72419896 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:54.634898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ebd53-eb7a-4877-b567-450f72419896 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:54.635600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ebd53-eb7a-4877-b567-450f72419896 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:54.635922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ebd53-eb7a-4877-b567-450f72419896 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:54.637587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ebd53-eb7a-4877-b567-450f72419896 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:54.650088 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 340/667] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:54.650964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ebd53-eb7a-4877-b567-450f72419896 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:54.658981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 328/668] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:54.660040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10863e60-8b5e-4668-a4db-df03ed551314 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:54.767918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ebd53-eb7a-4877-b567-450f72419896 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:54.769540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10863e60-8b5e-4668-a4db-df03ed551314 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:54.785739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:54.804735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192c491-29da-4e64-ac80-fd1bb5f71de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:54.804735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192c491-29da-4e64-ac80-fd1bb5f71de5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:54.804735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192c491-29da-4e64-ac80-fd1bb5f71de5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:54.804735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192c491-29da-4e64-ac80-fd1bb5f71de5 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:54.805201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192c491-29da-4e64-ac80-fd1bb5f71de5 None admin] RBAC: Adding query filter params (name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:54.805201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192c491-29da-4e64-ac80-fd1bb5f71de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:54.806140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192c491-29da-4e64-ac80-fd1bb5f71de5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:54.812247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cb048-976d-40ad-a879-b81b6590df41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:54.813587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cb048-976d-40ad-a879-b81b6590df41 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:54.813931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cb048-976d-40ad-a879-b81b6590df41 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:54.814270 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cb048-976d-40ad-a879-b81b6590df41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:54.820531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 341/669] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:54.820531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192c491-29da-4e64-ac80-fd1bb5f71de5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:54.831021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cb048-976d-40ad-a879-b81b6590df41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:54.832492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cb048-976d-40ad-a879-b81b6590df41 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-1a3cb048-976d-40ad-a879-b81b6590df41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:54.847906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:54.849066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:54.850134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:54.850134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:54.850134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:54.854191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 329/670] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:54.855087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cb048-976d-40ad-a879-b81b6590df41 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:54.864495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:54.925122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192c491-29da-4e64-ac80-fd1bb5f71de5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:54.963130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cb048-976d-40ad-a879-b81b6590df41 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:54.964616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554ea51-6cc1-4a76-ac98-ec16eb8b883c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:54.965220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554ea51-6cc1-4a76-ac98-ec16eb8b883c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:54.965498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554ea51-6cc1-4a76-ac98-ec16eb8b883c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:54.965768 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554ea51-6cc1-4a76-ac98-ec16eb8b883c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:54.966514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554ea51-6cc1-4a76-ac98-ec16eb8b883c None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=9628118c1df64fa4a7ea8ce37549d8b3) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:54.966824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554ea51-6cc1-4a76-ac98-ec16eb8b883c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:54.968255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554ea51-6cc1-4a76-ac98-ec16eb8b883c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:54.977883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 342/671] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:54 2022] GET /identity/v3/role_assignments?user.id=9628118c1df64fa4a7ea8ce37549d8b3&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:54.977883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554ea51-6cc1-4a76-ac98-ec16eb8b883c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.008909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca04f9d-316a-4be8-9de9-483888e53afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:55.009710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca04f9d-316a-4be8-9de9-483888e53afc None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:55.010103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca04f9d-316a-4be8-9de9-483888e53afc None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:55.010831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca04f9d-316a-4be8-9de9-483888e53afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:55.023144 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca04f9d-316a-4be8-9de9-483888e53afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:55.024898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca04f9d-316a-4be8-9de9-483888e53afc None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-dca04f9d-316a-4be8-9de9-483888e53afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:55.044008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:55.045453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:55.046545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:55.046545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:55.046545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:55.048253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:55.053548 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 330/672] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:55.054638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca04f9d-316a-4be8-9de9-483888e53afc None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.127984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554ea51-6cc1-4a76-ac98-ec16eb8b883c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:55.131307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:55.178783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00445a6d-0dca-473e-b498-6d7d73f17555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:55.179802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00445a6d-0dca-473e-b498-6d7d73f17555 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:55.180123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00445a6d-0dca-473e-b498-6d7d73f17555 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:55.180391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00445a6d-0dca-473e-b498-6d7d73f17555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:55.183622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca04f9d-316a-4be8-9de9-483888e53afc None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:55.195193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00445a6d-0dca-473e-b498-6d7d73f17555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:55.196667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00445a6d-0dca-473e-b498-6d7d73f17555 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-00445a6d-0dca-473e-b498-6d7d73f17555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:55.213438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:55.214445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:55.215393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:55.215393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:55.215393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:55.216161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 343/673] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:55.220055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00445a6d-0dca-473e-b498-6d7d73f17555 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.238751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654cc01-79f6-4050-b6b7-c48cc01d1072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:55.239361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654cc01-79f6-4050-b6b7-c48cc01d1072 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:55.239635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654cc01-79f6-4050-b6b7-c48cc01d1072 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:55.239904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654cc01-79f6-4050-b6b7-c48cc01d1072 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:55.240948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654cc01-79f6-4050-b6b7-c48cc01d1072 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:55.240948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654cc01-79f6-4050-b6b7-c48cc01d1072 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:55.242190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654cc01-79f6-4050-b6b7-c48cc01d1072 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:55.257155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 331/674] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:55.257864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654cc01-79f6-4050-b6b7-c48cc01d1072 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.332453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:55.339856 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00445a6d-0dca-473e-b498-6d7d73f17555 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:55.369654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654cc01-79f6-4050-b6b7-c48cc01d1072 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:55.380332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd97f8-9063-49c7-8303-7bf7c2013ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:55.381335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd97f8-9063-49c7-8303-7bf7c2013ca7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:55.381803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd97f8-9063-49c7-8303-7bf7c2013ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:55.382278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd97f8-9063-49c7-8303-7bf7c2013ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:55.383237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd97f8-9063-49c7-8303-7bf7c2013ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:55.383727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd97f8-9063-49c7-8303-7bf7c2013ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:55.385435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd97f8-9063-49c7-8303-7bf7c2013ca7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:55.392983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:55.400906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 344/675] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:55.402010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd97f8-9063-49c7-8303-7bf7c2013ca7 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.418253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640a1c7-ac1a-465f-931a-3bcaa07a8c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:55.419091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a1c7-ac1a-465f-931a-3bcaa07a8c5b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:55.419393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a1c7-ac1a-465f-931a-3bcaa07a8c5b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:55.419697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a1c7-ac1a-465f-931a-3bcaa07a8c5b None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:55.420871 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640a1c7-ac1a-465f-931a-3bcaa07a8c5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:55.421227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640a1c7-ac1a-465f-931a-3bcaa07a8c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:55.423508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640a1c7-ac1a-465f-931a-3bcaa07a8c5b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:55.442890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 332/676] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:55.444082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640a1c7-ac1a-465f-931a-3bcaa07a8c5b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.519943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd97f8-9063-49c7-8303-7bf7c2013ca7 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:55.559529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c04b5-216b-4fab-a872-a5d8a383910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:55.560172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c04b5-216b-4fab-a872-a5d8a383910c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:55.560443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c04b5-216b-4fab-a872-a5d8a383910c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:55.560707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c04b5-216b-4fab-a872-a5d8a383910c None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:55.568609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c04b5-216b-4fab-a872-a5d8a383910c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:55.570060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c04b5-216b-4fab-a872-a5d8a383910c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:55.578737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640a1c7-ac1a-465f-931a-3bcaa07a8c5b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-899c04b5-216b-4fab-a872-a5d8a383910c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:55.582376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:55.583430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:55.584329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:55.584329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:55.584329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:55.584329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:55.584329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:55.584329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:55.584329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:55.584329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:55.588241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 345/677] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:55.589115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c04b5-216b-4fab-a872-a5d8a383910c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.608179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:55.628066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f697c-2dad-4494-9c95-6310daa6dcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:55.628994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f697c-2dad-4494-9c95-6310daa6dcad None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:55.629475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f697c-2dad-4494-9c95-6310daa6dcad None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:55.629975 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f697c-2dad-4494-9c95-6310daa6dcad None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:55.637821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f697c-2dad-4494-9c95-6310daa6dcad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:55.640723 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f697c-2dad-4494-9c95-6310daa6dcad None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-341f697c-2dad-4494-9c95-6310daa6dcad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:55.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:55.655860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:55.656887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:55.656887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:55.656887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:55.656887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:55.656887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:55.656887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:55.656887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:55.656887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:55.659583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 333/678] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:55.660838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f697c-2dad-4494-9c95-6310daa6dcad None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.700656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c04b5-216b-4fab-a872-a5d8a383910c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:55.707565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:55.738787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216bd60-ff9e-4098-9558-4436ff1ddd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:55.739429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216bd60-ff9e-4098-9558-4436ff1ddd3c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:55.739701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216bd60-ff9e-4098-9558-4436ff1ddd3c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:55.739963 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216bd60-ff9e-4098-9558-4436ff1ddd3c None admin] PATH_INFO: `/v3/users/swift` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:55.749895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216bd60-ff9e-4098-9558-4436ff1ddd3c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:55.751509 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216bd60-ff9e-4098-9558-4436ff1ddd3c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-3216bd60-ff9e-4098-9558-4436ff1ddd3c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:55.763939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:55.764929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 346/679] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:05:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:55.767161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216bd60-ff9e-4098-9558-4436ff1ddd3c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.789344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f697c-2dad-4494-9c95-6310daa6dcad None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:55.830444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957793d-793f-443d-a43a-cefee0cc18f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:55.831421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957793d-793f-443d-a43a-cefee0cc18f5 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:55.832015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957793d-793f-443d-a43a-cefee0cc18f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:55.832485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957793d-793f-443d-a43a-cefee0cc18f5 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:55.833475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957793d-793f-443d-a43a-cefee0cc18f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:55.834081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957793d-793f-443d-a43a-cefee0cc18f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:55.837349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957793d-793f-443d-a43a-cefee0cc18f5 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:55.849828 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:55.853174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 334/680] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:55.854255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957793d-793f-443d-a43a-cefee0cc18f5 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.879618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216bd60-ff9e-4098-9558-4436ff1ddd3c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:55.914906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19121c82-ab89-4212-bd5d-04b3c09a5ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:55.915509 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19121c82-ab89-4212-bd5d-04b3c09a5ca2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:55.915783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19121c82-ab89-4212-bd5d-04b3c09a5ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:55.916054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19121c82-ab89-4212-bd5d-04b3c09a5ca2 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:55.916737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19121c82-ab89-4212-bd5d-04b3c09a5ca2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:55.917036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19121c82-ab89-4212-bd5d-04b3c09a5ca2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:55.918461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19121c82-ab89-4212-bd5d-04b3c09a5ca2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:55.932421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 347/681] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:55.933343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19121c82-ab89-4212-bd5d-04b3c09a5ca2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:55.967870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:55.972433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957793d-793f-443d-a43a-cefee0cc18f5 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:56.007748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc8c5c-660d-4a00-bf06-8acb9ea4548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:56.008353 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc8c5c-660d-4a00-bf06-8acb9ea4548f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:56.008621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc8c5c-660d-4a00-bf06-8acb9ea4548f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:56.008887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc8c5c-660d-4a00-bf06-8acb9ea4548f None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:56.009608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc8c5c-660d-4a00-bf06-8acb9ea4548f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:56.009908 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc8c5c-660d-4a00-bf06-8acb9ea4548f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:56.011359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc8c5c-660d-4a00-bf06-8acb9ea4548f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:56.037491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 335/682] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 22 17:05:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:56.037491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc8c5c-660d-4a00-bf06-8acb9ea4548f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:56.046569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19121c82-ab89-4212-bd5d-04b3c09a5ca2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:56.086732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4bedf6-f20f-4a96-b47c-255d3ad2e592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:56.087339 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bedf6-f20f-4a96-b47c-255d3ad2e592 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:56.087615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bedf6-f20f-4a96-b47c-255d3ad2e592 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:56.087884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bedf6-f20f-4a96-b47c-255d3ad2e592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:56.088612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bedf6-f20f-4a96-b47c-255d3ad2e592 None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=68088e59a2c64ff2948e0549f1fc7cb9) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:56.088913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bedf6-f20f-4a96-b47c-255d3ad2e592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:56.090352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bedf6-f20f-4a96-b47c-255d3ad2e592 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:56.093324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:56.098162 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 348/683] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:55 2022] GET /identity/v3/role_assignments?user.id=68088e59a2c64ff2948e0549f1fc7cb9&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:56.099440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d948670-60b2-4962-ba1e-5db907259ef5 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:56.099740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d948670-60b2-4962-ba1e-5db907259ef5 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:56.099998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d948670-60b2-4962-ba1e-5db907259ef5 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:56.101433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 349/684] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:56.106632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d948670-60b2-4962-ba1e-5db907259ef5 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:56.170468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc8c5c-660d-4a00-bf06-8acb9ea4548f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:56.224349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab85d8-37bc-4ef9-b872-d4f5f160412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:56.225672 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab85d8-37bc-4ef9-b872-d4f5f160412e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:56.226205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab85d8-37bc-4ef9-b872-d4f5f160412e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:56.227238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab85d8-37bc-4ef9-b872-d4f5f160412e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:56.228693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab85d8-37bc-4ef9-b872-d4f5f160412e None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=c29a91e099ba4cd5886fd910d978fb4d) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:56.229459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab85d8-37bc-4ef9-b872-d4f5f160412e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:56.231912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab85d8-37bc-4ef9-b872-d4f5f160412e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:56.238020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d948670-60b2-4962-ba1e-5db907259ef5 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:56.242250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 336/685] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:56 2022] GET /identity/v3/role_assignments?user.id=c29a91e099ba4cd5886fd910d978fb4d&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:56.244168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880c26a-6bf7-44c2-9b00-e76b08d854b9 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:56.244693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880c26a-6bf7-44c2-9b00-e76b08d854b9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:56.245176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880c26a-6bf7-44c2-9b00-e76b08d854b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:56.272102 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a880c26a-6bf7-44c2-9b00-e76b08d854b9 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:56.274885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a880c26a-6bf7-44c2-9b00-e76b08d854b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:56.284831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a897b5d-546d-4aa0-83fe-e2f260f039cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:56.285462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897b5d-546d-4aa0-83fe-e2f260f039cc None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:56.285789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897b5d-546d-4aa0-83fe-e2f260f039cc None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:56.286066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897b5d-546d-4aa0-83fe-e2f260f039cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:56.299924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:56.374878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:56.378936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a897b5d-546d-4aa0-83fe-e2f260f039cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:56.380596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a897b5d-546d-4aa0-83fe-e2f260f039cc None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:56.381067 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 337/686] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:56.511332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 350/687] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:56.517476 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880c26a-6bf7-44c2-9b00-e76b08d854b9 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:56.640056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880c26a-6bf7-44c2-9b00-e76b08d854b9 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:56.681143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8d7a2-79f7-4236-a3cc-f752c81a2969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:56.682143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8d7a2-79f7-4236-a3cc-f752c81a2969 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:56.682668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8d7a2-79f7-4236-a3cc-f752c81a2969 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:56.683167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8d7a2-79f7-4236-a3cc-f752c81a2969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:56.691069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a897b5d-546d-4aa0-83fe-e2f260f039cc None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:56.696835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8d7a2-79f7-4236-a3cc-f752c81a2969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:56.699547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8d7a2-79f7-4236-a3cc-f752c81a2969 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-aea8d7a2-79f7-4236-a3cc-f752c81a2969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:56.717127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:56.718148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:56.719139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:56.719139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:56.719139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:56.721668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 338/688] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:56.730830 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8d7a2-79f7-4236-a3cc-f752c81a2969 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:56.735912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:56.870404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:56.876185 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a897b5d-546d-4aa0-83fe-e2f260f039cc None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:56.936515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8d7a2-79f7-4236-a3cc-f752c81a2969 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:56.948659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40aa600-29c7-4b3a-9bc5-3ad0608bcfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:56.949651 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40aa600-29c7-4b3a-9bc5-3ad0608bcfde None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:56.950127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40aa600-29c7-4b3a-9bc5-3ad0608bcfde None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:56.950635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40aa600-29c7-4b3a-9bc5-3ad0608bcfde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:56.971316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40aa600-29c7-4b3a-9bc5-3ad0608bcfde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:56.973685 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40aa600-29c7-4b3a-9bc5-3ad0608bcfde None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-e40aa600-29c7-4b3a-9bc5-3ad0608bcfde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:57.004164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:57.005186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:57.006087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:57.006087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:57.006087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:57.007442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 351/689] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:57.013915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40aa600-29c7-4b3a-9bc5-3ad0608bcfde None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:57.022658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280c217-b3df-4dd8-87e1-d79f6673b444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:57.023288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280c217-b3df-4dd8-87e1-d79f6673b444 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:57.023565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280c217-b3df-4dd8-87e1-d79f6673b444 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:57.023838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280c217-b3df-4dd8-87e1-d79f6673b444 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:57.024500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280c217-b3df-4dd8-87e1-d79f6673b444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:57.024801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280c217-b3df-4dd8-87e1-d79f6673b444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:57.026185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280c217-b3df-4dd8-87e1-d79f6673b444 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:57.049284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 339/690] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:57.054937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280c217-b3df-4dd8-87e1-d79f6673b444 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:57.152126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40aa600-29c7-4b3a-9bc5-3ad0608bcfde None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:57.174831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:57.189506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f6f35-97f5-482f-921f-c43d4cdbfd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:57.195547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f6f35-97f5-482f-921f-c43d4cdbfd12 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:57.195547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f6f35-97f5-482f-921f-c43d4cdbfd12 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:57.195547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f6f35-97f5-482f-921f-c43d4cdbfd12 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:57.195547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f6f35-97f5-482f-921f-c43d4cdbfd12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:57.195547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f6f35-97f5-482f-921f-c43d4cdbfd12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:57.195547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f6f35-97f5-482f-921f-c43d4cdbfd12 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:57.205758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 352/691] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:57.210053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280c217-b3df-4dd8-87e1-d79f6673b444 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:57.211627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8dbe6-727e-4bd2-a697-07c8fbbfb881 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:57.211915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8dbe6-727e-4bd2-a697-07c8fbbfb881 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:57.212193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8dbe6-727e-4bd2-a697-07c8fbbfb881 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:57.213976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 353/692] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:57.214796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8dbe6-727e-4bd2-a697-07c8fbbfb881 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:57.242884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:57.248879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac6ac3-7976-4b85-865f-e0403aeae426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:57.249627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac6ac3-7976-4b85-865f-e0403aeae426 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:57.255044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac6ac3-7976-4b85-865f-e0403aeae426 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:57.255044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac6ac3-7976-4b85-865f-e0403aeae426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:57.258725 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac6ac3-7976-4b85-865f-e0403aeae426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:57.260303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac6ac3-7976-4b85-865f-e0403aeae426 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-31ac6ac3-7976-4b85-865f-e0403aeae426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:05:57.273444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:57.274686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:57.275793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:05:57.275793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:05:57.275793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:05:57.275793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:05:57.275793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:05:57.275793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:05:57.275793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:05:57.275793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:57.279193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 340/693] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:57.279193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1af79-697f-41a0-b3ee-e68036840b57 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:57.279193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1af79-697f-41a0-b3ee-e68036840b57 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:57.279557 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1af79-697f-41a0-b3ee-e68036840b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:57.299885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-20e1af79-697f-41a0-b3ee-e68036840b57 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:57.301255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-20e1af79-697f-41a0-b3ee-e68036840b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:57.328931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8dbe6-727e-4bd2-a697-07c8fbbfb881 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:57.367948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf4d31-2f9b-4af9-b606-43e890cb4118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:57.368717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf4d31-2f9b-4af9-b606-43e890cb4118 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:57.368999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf4d31-2f9b-4af9-b606-43e890cb4118 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:57.369335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf4d31-2f9b-4af9-b606-43e890cb4118 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:57.370007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf4d31-2f9b-4af9-b606-43e890cb4118 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:57.371587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf4d31-2f9b-4af9-b606-43e890cb4118 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:57.373821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 341/694] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:57.374877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1af79-697f-41a0-b3ee-e68036840b57 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:57.438232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:57.470755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 354/695] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:05:57 2022] POST /identity/v3/users => generated 254 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:05:57.494523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1af79-697f-41a0-b3ee-e68036840b57 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:57.533692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db1d9c-7f59-401e-8b8c-8b05b10453aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:57.534323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db1d9c-7f59-401e-8b8c-8b05b10453aa None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:57.534642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db1d9c-7f59-401e-8b8c-8b05b10453aa None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:57.534912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db1d9c-7f59-401e-8b8c-8b05b10453aa None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:57.535639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db1d9c-7f59-401e-8b8c-8b05b10453aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:57.535946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db1d9c-7f59-401e-8b8c-8b05b10453aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:57.537435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db1d9c-7f59-401e-8b8c-8b05b10453aa None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:57.553121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 342/696] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:57.559728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf4d31-2f9b-4af9-b606-43e890cb4118 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:57.673166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:57.708207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf4d31-2f9b-4af9-b606-43e890cb4118 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:57.747293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717797f-882a-4b81-baa3-0ea3c9abb570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:57.747914 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717797f-882a-4b81-baa3-0ea3c9abb570 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:57.748189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717797f-882a-4b81-baa3-0ea3c9abb570 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:57.748507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717797f-882a-4b81-baa3-0ea3c9abb570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:57.749291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717797f-882a-4b81-baa3-0ea3c9abb570 None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=38462b1165514ed7b948cba710782d4b) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:57.749649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717797f-882a-4b81-baa3-0ea3c9abb570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:57.752394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717797f-882a-4b81-baa3-0ea3c9abb570 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:57.759190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 355/697] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:05:57 2022] GET /identity/v3/role_assignments?user.id=38462b1165514ed7b948cba710782d4b&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:57.760945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db1d9c-7f59-401e-8b8c-8b05b10453aa None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:57.883476 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db1d9c-7f59-401e-8b8c-8b05b10453aa None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:57.902883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:57.927006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b470c4-4f18-4293-9360-06e8ecf5eb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:57.927647 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b470c4-4f18-4293-9360-06e8ecf5eb35 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:57.927922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b470c4-4f18-4293-9360-06e8ecf5eb35 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:57.928185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b470c4-4f18-4293-9360-06e8ecf5eb35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:57.940118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b470c4-4f18-4293-9360-06e8ecf5eb35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-57b470c4-4f18-4293-9360-06e8ecf5eb35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:05:57.944819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:57.945795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:57.946685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 343/698] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:05:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:57.953694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717797f-882a-4b81-baa3-0ea3c9abb570 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:58.071594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717797f-882a-4b81-baa3-0ea3c9abb570 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:58.104298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:58.109223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bba5e-5f04-42ab-96f1-9e7b416a38d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:58.110080 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bba5e-5f04-42ab-96f1-9e7b416a38d1 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:58.110456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bba5e-5f04-42ab-96f1-9e7b416a38d1 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:58.110769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bba5e-5f04-42ab-96f1-9e7b416a38d1 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:58.111444 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bba5e-5f04-42ab-96f1-9e7b416a38d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:58.111744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bba5e-5f04-42ab-96f1-9e7b416a38d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:58.113112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bba5e-5f04-42ab-96f1-9e7b416a38d1 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:58.123074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 356/699] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:05:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:05:58.128852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b470c4-4f18-4293-9360-06e8ecf5eb35 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:58.261579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b470c4-4f18-4293-9360-06e8ecf5eb35 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:58.306060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11460b-ba88-4d4c-8d80-2cb27e219ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:58.306740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11460b-ba88-4d4c-8d80-2cb27e219ad2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:58.307025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11460b-ba88-4d4c-8d80-2cb27e219ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:58.307299 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11460b-ba88-4d4c-8d80-2cb27e219ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:58.319475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11460b-ba88-4d4c-8d80-2cb27e219ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:58.320960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11460b-ba88-4d4c-8d80-2cb27e219ad2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-7e11460b-ba88-4d4c-8d80-2cb27e219ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:58.347232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:58.348345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:58.349253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:58.349253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:58.349253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:05:58.349253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 344/700] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:58.354396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bba5e-5f04-42ab-96f1-9e7b416a38d1 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:58.549345 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bba5e-5f04-42ab-96f1-9e7b416a38d1 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:58.603468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0347958-5397-4712-92c1-7e94bbbfd928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:58.603886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0347958-5397-4712-92c1-7e94bbbfd928 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:58.604260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0347958-5397-4712-92c1-7e94bbbfd928 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:58.604541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0347958-5397-4712-92c1-7e94bbbfd928 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:58.605060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05955f07-239d-48d0-9da4-a8ec942e15f0 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:58.605238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0347958-5397-4712-92c1-7e94bbbfd928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:58.605332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05955f07-239d-48d0-9da4-a8ec942e15f0 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:58.605538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0347958-5397-4712-92c1-7e94bbbfd928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:58.605661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05955f07-239d-48d0-9da4-a8ec942e15f0 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:58.606999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0347958-5397-4712-92c1-7e94bbbfd928 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:58.607384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 345/701] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:58.613687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f9b05-508d-438a-8ea2-330c74bae916 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:58.613966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f9b05-508d-438a-8ea2-330c74bae916 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:58.616433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f9b05-508d-438a-8ea2-330c74bae916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:58.636097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 357/702] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:58.642213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0347958-5397-4712-92c1-7e94bbbfd928 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:58.643387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-e39f9b05-508d-438a-8ea2-330c74bae916 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:58.647003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e39f9b05-508d-438a-8ea2-330c74bae916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:58.686463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:58.728597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:58.761928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 346/703] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 17:05:58.763283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c13990-6184-4f3e-9e45-dbd6c4a01da1 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:58.763580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c13990-6184-4f3e-9e45-dbd6c4a01da1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:58.763861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c13990-6184-4f3e-9e45-dbd6c4a01da1 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:58.769380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 347/704] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:05:58.771141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd90c0-f5c9-458c-961a-b058c6738e91 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:58.771257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd90c0-f5c9-458c-961a-b058c6738e91 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:58.771507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd90c0-f5c9-458c-961a-b058c6738e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:58.788313 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0347958-5397-4712-92c1-7e94bbbfd928 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:58.793588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1fbd90c0-f5c9-458c-961a-b058c6738e91 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:05:58.795206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1fbd90c0-f5c9-458c-961a-b058c6738e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:05:58.849933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99592-decc-4773-b20e-6ed14af1d275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:58.850918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99592-decc-4773-b20e-6ed14af1d275 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:58.851540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99592-decc-4773-b20e-6ed14af1d275 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:58.852867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99592-decc-4773-b20e-6ed14af1d275 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:58.867674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a99592-decc-4773-b20e-6ed14af1d275 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:58.869060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a99592-decc-4773-b20e-6ed14af1d275 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-33a99592-decc-4773-b20e-6ed14af1d275 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:05:58.890391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:58.892296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:58.893477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:58.893477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:58.893477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:05:58.893477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:05:58.893477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:05:58.893477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:05:58.893477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:05:58.893477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:05:58.893477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:05:58.893477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:58.897803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 358/705] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 22 17:05:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:58.904914 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99592-decc-4773-b20e-6ed14af1d275 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:58.918885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 348/706] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:05:59.044244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99592-decc-4773-b20e-6ed14af1d275 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:59.063997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:59.092886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88e236-0d6c-45e9-8392-5d107eb7d306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:59.093777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88e236-0d6c-45e9-8392-5d107eb7d306 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:59.094848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88e236-0d6c-45e9-8392-5d107eb7d306 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:59.095271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88e236-0d6c-45e9-8392-5d107eb7d306 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:59.096208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88e236-0d6c-45e9-8392-5d107eb7d306 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:59.096658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88e236-0d6c-45e9-8392-5d107eb7d306 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:59.098279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88e236-0d6c-45e9-8392-5d107eb7d306 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:59.122944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 359/707] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 22 17:05:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:59.128314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd90c0-f5c9-458c-961a-b058c6738e91 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:59.177054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88e236-0d6c-45e9-8392-5d107eb7d306 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:59.248834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:59.266854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd90c0-f5c9-458c-961a-b058c6738e91 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:59.304604 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb29b5-b1b8-4a13-a111-a27f2ec7abe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:59.305307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb29b5-b1b8-4a13-a111-a27f2ec7abe1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:59.305655 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb29b5-b1b8-4a13-a111-a27f2ec7abe1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:59.306000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb29b5-b1b8-4a13-a111-a27f2ec7abe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:59.314552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88e236-0d6c-45e9-8392-5d107eb7d306 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:59.361083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d67622-a7d3-4007-b731-f27005d68c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:59.361729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d67622-a7d3-4007-b731-f27005d68c59 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:59.362107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d67622-a7d3-4007-b731-f27005d68c59 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:59.362447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d67622-a7d3-4007-b731-f27005d68c59 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:59.363194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d67622-a7d3-4007-b731-f27005d68c59 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:59.364892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d67622-a7d3-4007-b731-f27005d68c59 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-98d67622-a7d3-4007-b731-f27005d68c59 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:59.377210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:59.378289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:59.379210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 360/708] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Aug 22 17:05:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:05:59.384517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d67622-a7d3-4007-b731-f27005d68c59 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:59.405274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb29b5-b1b8-4a13-a111-a27f2ec7abe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:59.408087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb29b5-b1b8-4a13-a111-a27f2ec7abe1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:59.428803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:59.511535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d67622-a7d3-4007-b731-f27005d68c59 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:59.524643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:59.549018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277752b-f50b-4f0c-abf0-9a53fd1298df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:59.549223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 349/709] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:05:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:05:59.549857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277752b-f50b-4f0c-abf0-9a53fd1298df None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:59.550248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277752b-f50b-4f0c-abf0-9a53fd1298df None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:59.550619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277752b-f50b-4f0c-abf0-9a53fd1298df None admin] PATH_INFO: `/v3/services/network` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:59.550876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb29b5-b1b8-4a13-a111-a27f2ec7abe1 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:59.551442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277752b-f50b-4f0c-abf0-9a53fd1298df None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:59.553431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277752b-f50b-4f0c-abf0-9a53fd1298df None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-8277752b-f50b-4f0c-abf0-9a53fd1298df None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:59.564664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:59.565764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:59.566643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 361/710] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:05:59 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:59.567440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277752b-f50b-4f0c-abf0-9a53fd1298df None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:59.686172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb29b5-b1b8-4a13-a111-a27f2ec7abe1 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:59.696315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:59.701616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277752b-f50b-4f0c-abf0-9a53fd1298df None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:59.723913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a3d39-41bc-4d1c-897e-a233d8ca3b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:59.724581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a3d39-41bc-4d1c-897e-a233d8ca3b5d None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:59.724927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a3d39-41bc-4d1c-897e-a233d8ca3b5d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:59.725263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a3d39-41bc-4d1c-897e-a233d8ca3b5d None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:59.725963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a3d39-41bc-4d1c-897e-a233d8ca3b5d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:59.726336 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a3d39-41bc-4d1c-897e-a233d8ca3b5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:59.727803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a3d39-41bc-4d1c-897e-a233d8ca3b5d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:59.738162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 350/711] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 22 17:05:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:59.739206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a3d39-41bc-4d1c-897e-a233d8ca3b5d None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:59.747605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f16b60-0c15-4be5-8e0a-0aec3f5cf311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:59.748512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f16b60-0c15-4be5-8e0a-0aec3f5cf311 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:59.749517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f16b60-0c15-4be5-8e0a-0aec3f5cf311 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:59.750006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f16b60-0c15-4be5-8e0a-0aec3f5cf311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:59.767082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f16b60-0c15-4be5-8e0a-0aec3f5cf311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:59.769799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f16b60-0c15-4be5-8e0a-0aec3f5cf311 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:59.793603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-37f16b60-0c15-4be5-8e0a-0aec3f5cf311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:05:59.795522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:05:59.796471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:05:59.798164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:05:59.798164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:05:59.798164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:05:59.800915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 362/712] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:05:59.802034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f16b60-0c15-4be5-8e0a-0aec3f5cf311 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:59.867172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a3d39-41bc-4d1c-897e-a233d8ca3b5d None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:59.914625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b80a83-5f2c-413c-9883-8a97a8af8884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:59.915589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b80a83-5f2c-413c-9883-8a97a8af8884 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:59.915951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b80a83-5f2c-413c-9883-8a97a8af8884 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:59.916376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b80a83-5f2c-413c-9883-8a97a8af8884 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:59.917080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b80a83-5f2c-413c-9883-8a97a8af8884 None admin] RBAC: Adding query filter params (name=network) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:59.917455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b80a83-5f2c-413c-9883-8a97a8af8884 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:59.919038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b80a83-5f2c-413c-9883-8a97a8af8884 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:59.923147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f16b60-0c15-4be5-8e0a-0aec3f5cf311 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:05:59.929281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 351/713] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:05:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:59.930274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b80a83-5f2c-413c-9883-8a97a8af8884 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:05:59.961695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfee59b-b96a-4b1c-9496-296570664a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:05:59.962389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfee59b-b96a-4b1c-9496-296570664a70 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:05:59.962703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfee59b-b96a-4b1c-9496-296570664a70 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:05:59.962983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfee59b-b96a-4b1c-9496-296570664a70 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:05:59.963724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfee59b-b96a-4b1c-9496-296570664a70 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:05:59.964034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfee59b-b96a-4b1c-9496-296570664a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:05:59.965679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfee59b-b96a-4b1c-9496-296570664a70 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:05:59.973052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 363/714] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 22 17:05:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:05:59.974038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfee59b-b96a-4b1c-9496-296570664a70 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:00.066352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:00.079117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b80a83-5f2c-413c-9883-8a97a8af8884 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:00.097834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:00.136463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359c9e0-2fc0-4eef-9690-4402b44e82a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:00.136853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c9e0-2fc0-4eef-9690-4402b44e82a7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:00.137133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c9e0-2fc0-4eef-9690-4402b44e82a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:00.137406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c9e0-2fc0-4eef-9690-4402b44e82a7 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:00.138876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359c9e0-2fc0-4eef-9690-4402b44e82a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:00.139141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359c9e0-2fc0-4eef-9690-4402b44e82a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:00.140538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359c9e0-2fc0-4eef-9690-4402b44e82a7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:00.158517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 352/715] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:00.159478 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f04fa0-97ac-4cf8-a716-2ef6cc808378 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:00.159764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f04fa0-97ac-4cf8-a716-2ef6cc808378 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:00.160040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f04fa0-97ac-4cf8-a716-2ef6cc808378 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:00.161432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 353/716] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:00.166329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f04fa0-97ac-4cf8-a716-2ef6cc808378 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:00.177672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfee59b-b96a-4b1c-9496-296570664a70 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:00.222985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf2ed0-44b0-44bb-8694-6c5f540cab5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:00.223514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf2ed0-44b0-44bb-8694-6c5f540cab5c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:00.223837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf2ed0-44b0-44bb-8694-6c5f540cab5c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:00.224143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf2ed0-44b0-44bb-8694-6c5f540cab5c None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:00.224795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf2ed0-44b0-44bb-8694-6c5f540cab5c None admin] RBAC: Adding query filter params (type=network) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:00.225581 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf2ed0-44b0-44bb-8694-6c5f540cab5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:00.227329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf2ed0-44b0-44bb-8694-6c5f540cab5c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:00.234436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 364/717] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:05:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:00.236721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87039b-94eb-4506-bd69-00a3ae43871e None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:00.237051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87039b-94eb-4506-bd69-00a3ae43871e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:00.237326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87039b-94eb-4506-bd69-00a3ae43871e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:00.263188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1f87039b-94eb-4506-bd69-00a3ae43871e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:00.264661 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1f87039b-94eb-4506-bd69-00a3ae43871e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:00.314375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f04fa0-97ac-4cf8-a716-2ef6cc808378 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:00.348829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 365/718] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:06:00.350213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03729905-1c5f-4cb5-a7c4-0d58d7b26134 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:00.352099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03729905-1c5f-4cb5-a7c4-0d58d7b26134 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:00.352099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03729905-1c5f-4cb5-a7c4-0d58d7b26134 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:00.352548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 366/719] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:00.358466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7a7c2-eebd-4c85-83aa-69755e3ad7c4 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:00.358817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7a7c2-eebd-4c85-83aa-69755e3ad7c4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:00.359332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7a7c2-eebd-4c85-83aa-69755e3ad7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:00.382036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be33215-f796-4988-aee3-bf1b95b9c4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:00.382738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be33215-f796-4988-aee3-bf1b95b9c4b6 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:00.383014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be33215-f796-4988-aee3-bf1b95b9c4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:00.387866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be33215-f796-4988-aee3-bf1b95b9c4b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:00.392805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be33215-f796-4988-aee3-bf1b95b9c4b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:00.394447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be33215-f796-4988-aee3-bf1b95b9c4b6 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:00.399105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e7d7a7c2-eebd-4c85-83aa-69755e3ad7c4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:00.400533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e7d7a7c2-eebd-4c85-83aa-69755e3ad7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-7be33215-f796-4988-aee3-bf1b95b9c4b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:00.408428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:00.410215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:00.411368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:00.411368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:00.411368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:00.411368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:00.411368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:00.411368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:00.411368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:00.411368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:00.411808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:00.411889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 354/720] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:00.421641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be33215-f796-4988-aee3-bf1b95b9c4b6 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:00.470866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:00.507339 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 367/721] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:06:00.561344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be33215-f796-4988-aee3-bf1b95b9c4b6 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:00.616691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3198b5-ac18-42d2-8659-7563f1299206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:00.617677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3198b5-ac18-42d2-8659-7563f1299206 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:00.618150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3198b5-ac18-42d2-8659-7563f1299206 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:00.618636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3198b5-ac18-42d2-8659-7563f1299206 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:00.619532 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3198b5-ac18-42d2-8659-7563f1299206 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:00.620027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3198b5-ac18-42d2-8659-7563f1299206 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:00.621690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3198b5-ac18-42d2-8659-7563f1299206 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:00.638754 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 355/722] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:00.645598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7a7c2-eebd-4c85-83aa-69755e3ad7c4 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:00.706989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3198b5-ac18-42d2-8659-7563f1299206 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:00.829649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7a7c2-eebd-4c85-83aa-69755e3ad7c4 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:00.869907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:00.878896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfe9b8-4201-44f7-a91b-a6bba13f6775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:00.879627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfe9b8-4201-44f7-a91b-a6bba13f6775 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:00.879902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfe9b8-4201-44f7-a91b-a6bba13f6775 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:00.885201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfe9b8-4201-44f7-a91b-a6bba13f6775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:00.885310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3198b5-ac18-42d2-8659-7563f1299206 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:00.885389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfe9b8-4201-44f7-a91b-a6bba13f6775 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=9628118c1df64fa4a7ea8ce37549d8b3) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:00.885389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfe9b8-4201-44f7-a91b-a6bba13f6775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:00.885389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfe9b8-4201-44f7-a91b-a6bba13f6775 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:00.894900 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 368/723] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:00 2022] GET /identity/v3/role_assignments?user.id=9628118c1df64fa4a7ea8ce37549d8b3&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=4b10782d99b74f5bb453b82286012e0c => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:00.895836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfe9b8-4201-44f7-a91b-a6bba13f6775 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:00.905070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:00.945206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00e0f6-729b-4b7d-a354-c9705fdb5b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:00.945850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00e0f6-729b-4b7d-a354-c9705fdb5b62 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:00.946130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00e0f6-729b-4b7d-a354-c9705fdb5b62 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:00.946398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00e0f6-729b-4b7d-a354-c9705fdb5b62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:00.963668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00e0f6-729b-4b7d-a354-c9705fdb5b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-9e00e0f6-729b-4b7d-a354-c9705fdb5b62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:00.968781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:00.970341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:00.971257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 356/724] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:06:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:00.982510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00e0f6-729b-4b7d-a354-c9705fdb5b62 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:01.097441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfe9b8-4201-44f7-a91b-a6bba13f6775 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:01.144298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00e0f6-729b-4b7d-a354-c9705fdb5b62 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:01.149414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e227b-414c-4737-9f99-eb343efe8da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:01.150117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e227b-414c-4737-9f99-eb343efe8da2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:01.150402 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e227b-414c-4737-9f99-eb343efe8da2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:01.150696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e227b-414c-4737-9f99-eb343efe8da2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:01.175426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e227b-414c-4737-9f99-eb343efe8da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-f29e227b-414c-4737-9f99-eb343efe8da2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:01.189908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:01.191138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:01.192260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 369/725] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:01.210120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e227b-414c-4737-9f99-eb343efe8da2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:01.224185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2d79b-20bb-4a9d-80c1-8565a361a684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:01.225068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2d79b-20bb-4a9d-80c1-8565a361a684 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:01.225521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2d79b-20bb-4a9d-80c1-8565a361a684 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:01.225937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2d79b-20bb-4a9d-80c1-8565a361a684 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:01.226803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2d79b-20bb-4a9d-80c1-8565a361a684 None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:01.227255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2d79b-20bb-4a9d-80c1-8565a361a684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:01.228856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2d79b-20bb-4a9d-80c1-8565a361a684 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:01.248937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 357/726] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:06:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:01.267086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2d79b-20bb-4a9d-80c1-8565a361a684 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:01.290507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:01.316085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:01.334608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e227b-414c-4737-9f99-eb343efe8da2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:01.378455 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f2820-ab21-45d4-87b2-d4dce5f35d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:01.379067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f2820-ab21-45d4-87b2-d4dce5f35d89 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:01.379346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f2820-ab21-45d4-87b2-d4dce5f35d89 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:01.379666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f2820-ab21-45d4-87b2-d4dce5f35d89 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:01.380346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f2820-ab21-45d4-87b2-d4dce5f35d89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:01.380649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f2820-ab21-45d4-87b2-d4dce5f35d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:01.382025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f2820-ab21-45d4-87b2-d4dce5f35d89 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:01.396454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 370/727] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:06:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:01.397780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f2820-ab21-45d4-87b2-d4dce5f35d89 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:01.655746 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2d79b-20bb-4a9d-80c1-8565a361a684 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:01.677994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f2820-ab21-45d4-87b2-d4dce5f35d89 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:01.697454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d77b67-0a6c-4c88-adc2-045f1037b31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:01.698550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d77b67-0a6c-4c88-adc2-045f1037b31b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:01.698836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d77b67-0a6c-4c88-adc2-045f1037b31b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:01.699110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d77b67-0a6c-4c88-adc2-045f1037b31b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:01.712297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d77b67-0a6c-4c88-adc2-045f1037b31b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:01.713760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d77b67-0a6c-4c88-adc2-045f1037b31b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-67d77b67-0a6c-4c88-adc2-045f1037b31b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:01.731239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:01.732247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:01.733142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:01.733142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:01.733142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:01.733324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:01.740192 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 358/728] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:01.741502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d77b67-0a6c-4c88-adc2-045f1037b31b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:01.776144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c65f7-d864-42d3-9f87-5134ecc1ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:01.776858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c65f7-d864-42d3-9f87-5134ecc1ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:01.777133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c65f7-d864-42d3-9f87-5134ecc1ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:01.777405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c65f7-d864-42d3-9f87-5134ecc1ddc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:01.807305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c65f7-d864-42d3-9f87-5134ecc1ddc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:01.808783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c65f7-d864-42d3-9f87-5134ecc1ddc7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:01.816671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-843c65f7-d864-42d3-9f87-5134ecc1ddc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:01.836503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:01.838907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:01.841044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:01.841044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:01.841044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:01.846951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 371/729] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 449 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:01.863255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c65f7-d864-42d3-9f87-5134ecc1ddc7 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:01.864128 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d77b67-0a6c-4c88-adc2-045f1037b31b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:01.905455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b5d52-0183-4468-9bde-8c4228009dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:01.906316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b5d52-0183-4468-9bde-8c4228009dd0 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:01.907104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b5d52-0183-4468-9bde-8c4228009dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:01.909265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b5d52-0183-4468-9bde-8c4228009dd0 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:01.909265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b5d52-0183-4468-9bde-8c4228009dd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:01.909265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b5d52-0183-4468-9bde-8c4228009dd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:01.909941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b5d52-0183-4468-9bde-8c4228009dd0 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:01.928816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 359/730] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:01.930328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b5d52-0183-4468-9bde-8c4228009dd0 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:02.062397 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c65f7-d864-42d3-9f87-5134ecc1ddc7 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:02.094084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b5d52-0183-4468-9bde-8c4228009dd0 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:02.100825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a30835-af5a-4fb8-a3a3-f9f0aa582803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:02.101430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a30835-af5a-4fb8-a3a3-f9f0aa582803 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.101707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a30835-af5a-4fb8-a3a3-f9f0aa582803 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.101977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a30835-af5a-4fb8-a3a3-f9f0aa582803 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.102783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a30835-af5a-4fb8-a3a3-f9f0aa582803 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:02.102984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a30835-af5a-4fb8-a3a3-f9f0aa582803 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:02.105343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a30835-af5a-4fb8-a3a3-f9f0aa582803 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:02.124949 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 372/731] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:02.127007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a30835-af5a-4fb8-a3a3-f9f0aa582803 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:02.158988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:02.173760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:02.187325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaab1bd-c743-4b1e-9da3-29c2ed150a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:02.188615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab1bd-c743-4b1e-9da3-29c2ed150a16 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.188896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab1bd-c743-4b1e-9da3-29c2ed150a16 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.189156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab1bd-c743-4b1e-9da3-29c2ed150a16 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.207623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaab1bd-c743-4b1e-9da3-29c2ed150a16 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:02.209086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaab1bd-c743-4b1e-9da3-29c2ed150a16 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-fcaab1bd-c743-4b1e-9da3-29c2ed150a16 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:02.240139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:02.242380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:02.244776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 360/732] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 22 17:06:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:02.246215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a30835-af5a-4fb8-a3a3-f9f0aa582803 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:02.256052 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaab1bd-c743-4b1e-9da3-29c2ed150a16 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:02.287390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226d93b-49af-446d-af91-c16f65388425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:02.288416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226d93b-49af-446d-af91-c16f65388425 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.288720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226d93b-49af-446d-af91-c16f65388425 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.289165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226d93b-49af-446d-af91-c16f65388425 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.301292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226d93b-49af-446d-af91-c16f65388425 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:02.303352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226d93b-49af-446d-af91-c16f65388425 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-4226d93b-49af-446d-af91-c16f65388425 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:02.319935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:02.322236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:02.324704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:02.324704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:02.324704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:02.324704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:02.324704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:02.324704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:02.324704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:02.324704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:02.324704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:06:02.324704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:02.326037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 373/733] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:02.329930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537d1ef-cbcb-4d52-a8a1-7256c6904cd2 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.329930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537d1ef-cbcb-4d52-a8a1-7256c6904cd2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.329930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537d1ef-cbcb-4d52-a8a1-7256c6904cd2 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.332872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 374/734] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:02.332872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537d1ef-cbcb-4d52-a8a1-7256c6904cd2 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:02.395019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaab1bd-c743-4b1e-9da3-29c2ed150a16 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:02.436611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1666a3d-affb-4961-bcd3-8eb1abd8d5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:02.438432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1666a3d-affb-4961-bcd3-8eb1abd8d5aa None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.439036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1666a3d-affb-4961-bcd3-8eb1abd8d5aa None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.439592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1666a3d-affb-4961-bcd3-8eb1abd8d5aa None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.440475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1666a3d-affb-4961-bcd3-8eb1abd8d5aa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:02.441062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1666a3d-affb-4961-bcd3-8eb1abd8d5aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:02.443384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1666a3d-affb-4961-bcd3-8eb1abd8d5aa None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:02.459741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:02.465361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537d1ef-cbcb-4d52-a8a1-7256c6904cd2 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:02.471340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 361/735] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 22 17:06:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:02.473534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2549-ca7a-4f11-98ea-ef5806f93553 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.473534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2549-ca7a-4f11-98ea-ef5806f93553 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.473534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2549-ca7a-4f11-98ea-ef5806f93553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.492916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-337b2549-ca7a-4f11-98ea-ef5806f93553 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:02.494198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-337b2549-ca7a-4f11-98ea-ef5806f93553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:02.512666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b138f59-7aaf-4cf0-a96c-0cbd41483a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:02.513586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b138f59-7aaf-4cf0-a96c-0cbd41483a08 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.513869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b138f59-7aaf-4cf0-a96c-0cbd41483a08 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.514178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b138f59-7aaf-4cf0-a96c-0cbd41483a08 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.514939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b138f59-7aaf-4cf0-a96c-0cbd41483a08 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:02.515248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b138f59-7aaf-4cf0-a96c-0cbd41483a08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:02.517238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b138f59-7aaf-4cf0-a96c-0cbd41483a08 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:02.535190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 375/736] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 22 17:06:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:02.536077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b138f59-7aaf-4cf0-a96c-0cbd41483a08 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:02.549568 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:02.582523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 362/737] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:06:02.582523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fcc31-4013-4b8d-918b-c7946f5c78a8 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.582523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fcc31-4013-4b8d-918b-c7946f5c78a8 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.582523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fcc31-4013-4b8d-918b-c7946f5c78a8 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.582523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 363/738] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:02.582523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fcc31-4013-4b8d-918b-c7946f5c78a8 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:02.672736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b138f59-7aaf-4cf0-a96c-0cbd41483a08 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:02.708703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fcc31-4013-4b8d-918b-c7946f5c78a8 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:02.715111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd7ee8-c778-4bed-b6ad-329dd850724a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:02.715980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd7ee8-c778-4bed-b6ad-329dd850724a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.716431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd7ee8-c778-4bed-b6ad-329dd850724a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.716876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd7ee8-c778-4bed-b6ad-329dd850724a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.746950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:02.749168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df09a7-e2e2-4bcf-8c07-62e4a5891769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:02.749779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df09a7-e2e2-4bcf-8c07-62e4a5891769 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.750202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df09a7-e2e2-4bcf-8c07-62e4a5891769 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.750503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df09a7-e2e2-4bcf-8c07-62e4a5891769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.796522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:02.796974 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd7ee8-c778-4bed-b6ad-329dd850724a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:02.799175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd7ee8-c778-4bed-b6ad-329dd850724a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:02.822516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df09a7-e2e2-4bcf-8c07-62e4a5891769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:02.823995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df09a7-e2e2-4bcf-8c07-62e4a5891769 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:02.910071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 376/739] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:06:02.911737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb34b62-e484-4d94-8ef9-5253a92d85d4 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:02.912023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb34b62-e484-4d94-8ef9-5253a92d85d4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:02.912341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb34b62-e484-4d94-8ef9-5253a92d85d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:02.931200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-dcb34b62-e484-4d94-8ef9-5253a92d85d4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:02.932372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dcb34b62-e484-4d94-8ef9-5253a92d85d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:02.934927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 364/740] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 17:06:02.936311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df09a7-e2e2-4bcf-8c07-62e4a5891769 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:02.989301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:03.007542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 377/741] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:06:03.008299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb34b62-e484-4d94-8ef9-5253a92d85d4 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:03.020769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:03.055748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df09a7-e2e2-4bcf-8c07-62e4a5891769 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:03.094968 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71152cd-7a52-4a7f-a355-d5a91d752869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:03.095574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71152cd-7a52-4a7f-a355-d5a91d752869 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:03.095853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71152cd-7a52-4a7f-a355-d5a91d752869 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:03.096416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71152cd-7a52-4a7f-a355-d5a91d752869 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:03.097059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71152cd-7a52-4a7f-a355-d5a91d752869 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:03.098424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71152cd-7a52-4a7f-a355-d5a91d752869 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:03.117175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb34b62-e484-4d94-8ef9-5253a92d85d4 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:03.151711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52d1fa-5005-4f72-800e-dd3c5498b83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:03.152338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52d1fa-5005-4f72-800e-dd3c5498b83b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:03.152615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52d1fa-5005-4f72-800e-dd3c5498b83b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:03.152882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52d1fa-5005-4f72-800e-dd3c5498b83b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:03.163014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52d1fa-5005-4f72-800e-dd3c5498b83b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:03.164447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52d1fa-5005-4f72-800e-dd3c5498b83b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ed52d1fa-5005-4f72-800e-dd3c5498b83b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:03.179048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:03.180059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:03.180976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:03.180976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:03.180976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:03.180976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 378/742] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:03.180976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52d1fa-5005-4f72-800e-dd3c5498b83b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:03.190688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 365/743] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:06:02 2022] POST /identity/v3/services => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:03.192605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43e1c9-0399-43ae-94a9-c7eb9db11645 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:03.193939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43e1c9-0399-43ae-94a9-c7eb9db11645 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:03.194442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43e1c9-0399-43ae-94a9-c7eb9db11645 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:03.197047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 366/744] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:03.198688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43e1c9-0399-43ae-94a9-c7eb9db11645 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:03.243100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:03.316868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52d1fa-5005-4f72-800e-dd3c5498b83b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:03.351637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43e1c9-0399-43ae-94a9-c7eb9db11645 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:03.361568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05903e-0c7d-4114-b6ec-4c5eff32bd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:03.362225 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05903e-0c7d-4114-b6ec-4c5eff32bd3b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:03.362533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05903e-0c7d-4114-b6ec-4c5eff32bd3b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:03.362818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05903e-0c7d-4114-b6ec-4c5eff32bd3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:03.374583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05903e-0c7d-4114-b6ec-4c5eff32bd3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:03.376769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05903e-0c7d-4114-b6ec-4c5eff32bd3b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:03.390060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ad05903e-0c7d-4114-b6ec-4c5eff32bd3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:03.392445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:03.393419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:03.394343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:03.394343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:03.394343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:03.394343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 379/745] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:03.395420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579eb82-4b42-40c7-a248-9f3a990114d7 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:03.395705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579eb82-4b42-40c7-a248-9f3a990114d7 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:03.395999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579eb82-4b42-40c7-a248-9f3a990114d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:03.396333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebb15f-8c1f-4eae-bce6-defd78e2a96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:03.396941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebb15f-8c1f-4eae-bce6-defd78e2a96e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:03.397222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebb15f-8c1f-4eae-bce6-defd78e2a96e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:03.397492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebb15f-8c1f-4eae-bce6-defd78e2a96e None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:03.398139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebb15f-8c1f-4eae-bce6-defd78e2a96e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:03.398632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebb15f-8c1f-4eae-bce6-defd78e2a96e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:03.400085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebb15f-8c1f-4eae-bce6-defd78e2a96e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:03.413309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 367/746] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:03.415423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebb15f-8c1f-4eae-bce6-defd78e2a96e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:03.415863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-9579eb82-4b42-40c7-a248-9f3a990114d7 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:03.417049 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9579eb82-4b42-40c7-a248-9f3a990114d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:03.488090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 380/747] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 17:06:03.488955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579eb82-4b42-40c7-a248-9f3a990114d7 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:03.526026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:03.541735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebb15f-8c1f-4eae-bce6-defd78e2a96e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:03.578447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b445f67-78c2-4d72-96d6-4c1261e7c34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:03.579098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b445f67-78c2-4d72-96d6-4c1261e7c34b None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:03.579389 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b445f67-78c2-4d72-96d6-4c1261e7c34b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:03.579636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b445f67-78c2-4d72-96d6-4c1261e7c34b None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:03.580442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b445f67-78c2-4d72-96d6-4c1261e7c34b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:03.581679 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b445f67-78c2-4d72-96d6-4c1261e7c34b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:03.615920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579eb82-4b42-40c7-a248-9f3a990114d7 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:03.653519 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078231f-a59b-4d25-9c94-f76f984f56ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:03.654433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078231f-a59b-4d25-9c94-f76f984f56ef None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:03.654883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078231f-a59b-4d25-9c94-f76f984f56ef None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:03.655326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078231f-a59b-4d25-9c94-f76f984f56ef None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:03.656233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078231f-a59b-4d25-9c94-f76f984f56ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:03.656829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078231f-a59b-4d25-9c94-f76f984f56ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:03.658386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078231f-a59b-4d25-9c94-f76f984f56ef None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:03.671098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 381/748] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:03.672699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078231f-a59b-4d25-9c94-f76f984f56ef None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:03.690805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 368/749] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:06:03 2022] POST /identity/v3/services => generated 240 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:03.691829 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b445f67-78c2-4d72-96d6-4c1261e7c34b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:03.767647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:03.788342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:03.792646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078231f-a59b-4d25-9c94-f76f984f56ef None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:03.847153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90343769-3542-4e34-a8ac-81ccf397a92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:03.848060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90343769-3542-4e34-a8ac-81ccf397a92f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:03.848485 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90343769-3542-4e34-a8ac-81ccf397a92f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:03.848925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90343769-3542-4e34-a8ac-81ccf397a92f None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:03.852823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b445f67-78c2-4d72-96d6-4c1261e7c34b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:03.857539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90343769-3542-4e34-a8ac-81ccf397a92f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:03.859595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90343769-3542-4e34-a8ac-81ccf397a92f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-90343769-3542-4e34-a8ac-81ccf397a92f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:03.871811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:03.872812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:03.873985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:03.873985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:03.873985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:03.873985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:03.873985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:03.873985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:03.873985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:03.873985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:03.873985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 382/750] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:03.876259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90343769-3542-4e34-a8ac-81ccf397a92f None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:03.903181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed517b-c736-4c75-adec-5b1346f8bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:03.904019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed517b-c736-4c75-adec-5b1346f8bc51 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:03.904617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed517b-c736-4c75-adec-5b1346f8bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:03.904776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed517b-c736-4c75-adec-5b1346f8bc51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:03.917398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed517b-c736-4c75-adec-5b1346f8bc51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:03.920754 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed517b-c736-4c75-adec-5b1346f8bc51 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-5aed517b-c736-4c75-adec-5b1346f8bc51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:03.936227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:03.938315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:03.939425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:03.939425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:03.939425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:03.939425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:03.939425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:03.939425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:03.939425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:03.939425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:03.940695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 369/751] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:03.945541 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed517b-c736-4c75-adec-5b1346f8bc51 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:04.004126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90343769-3542-4e34-a8ac-81ccf397a92f None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:04.039460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568599c2-2c5d-4673-8d74-2391d9c1c580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:04.040231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568599c2-2c5d-4673-8d74-2391d9c1c580 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:04.040607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568599c2-2c5d-4673-8d74-2391d9c1c580 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:04.040972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568599c2-2c5d-4673-8d74-2391d9c1c580 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:04.047403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:04.055795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568599c2-2c5d-4673-8d74-2391d9c1c580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-568599c2-2c5d-4673-8d74-2391d9c1c580 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:04.060822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:04.062855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:04.063881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:04.065092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 383/752] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:04.066190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568599c2-2c5d-4673-8d74-2391d9c1c580 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:04.080226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed517b-c736-4c75-adec-5b1346f8bc51 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:04.091468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:04.117022 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7308cb-d52e-47ed-aa56-ffae79576b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:04.117781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7308cb-d52e-47ed-aa56-ffae79576b46 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:04.118155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7308cb-d52e-47ed-aa56-ffae79576b46 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:04.118560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7308cb-d52e-47ed-aa56-ffae79576b46 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:04.119359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7308cb-d52e-47ed-aa56-ffae79576b46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:04.119766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7308cb-d52e-47ed-aa56-ffae79576b46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:04.121315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7308cb-d52e-47ed-aa56-ffae79576b46 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:04.135859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 370/753] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:04.136925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7308cb-d52e-47ed-aa56-ffae79576b46 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:04.194958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568599c2-2c5d-4673-8d74-2391d9c1c580 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:04.235449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98823405-c0f7-4f6c-98fe-5fca6733774b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:04.236238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98823405-c0f7-4f6c-98fe-5fca6733774b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:04.236666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98823405-c0f7-4f6c-98fe-5fca6733774b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:04.237025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98823405-c0f7-4f6c-98fe-5fca6733774b None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:04.237929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98823405-c0f7-4f6c-98fe-5fca6733774b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:04.238333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98823405-c0f7-4f6c-98fe-5fca6733774b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:04.239895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98823405-c0f7-4f6c-98fe-5fca6733774b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:04.254247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 384/754] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:04.255374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98823405-c0f7-4f6c-98fe-5fca6733774b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:04.266700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7308cb-d52e-47ed-aa56-ffae79576b46 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:04.306947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391dc79-137b-4dc1-b32c-900108cd144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:04.307472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391dc79-137b-4dc1-b32c-900108cd144e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:04.307851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391dc79-137b-4dc1-b32c-900108cd144e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:04.308123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391dc79-137b-4dc1-b32c-900108cd144e None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:04.308895 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391dc79-137b-4dc1-b32c-900108cd144e None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:04.309207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391dc79-137b-4dc1-b32c-900108cd144e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:04.310698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391dc79-137b-4dc1-b32c-900108cd144e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:04.315845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:04.322014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 371/755] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:06:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:04.323141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391dc79-137b-4dc1-b32c-900108cd144e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:04.361101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:04.364133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98823405-c0f7-4f6c-98fe-5fca6733774b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:04.401861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b31291-a68f-417e-bbf7-3276ad0f3769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:04.402614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b31291-a68f-417e-bbf7-3276ad0f3769 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:04.402996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b31291-a68f-417e-bbf7-3276ad0f3769 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:04.403358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b31291-a68f-417e-bbf7-3276ad0f3769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:04.404160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b31291-a68f-417e-bbf7-3276ad0f3769 None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=9001ee719e294f79b35f13389ac9cbaa) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:04.404532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b31291-a68f-417e-bbf7-3276ad0f3769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:04.406054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b31291-a68f-417e-bbf7-3276ad0f3769 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:04.418509 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 385/756] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:04 2022] GET /identity/v3/role_assignments?user.id=9001ee719e294f79b35f13389ac9cbaa&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:04.418509 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b31291-a68f-417e-bbf7-3276ad0f3769 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:04.482430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391dc79-137b-4dc1-b32c-900108cd144e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:04.555851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd40dc-935e-4abd-9aae-2e415cd247f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:04.556764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd40dc-935e-4abd-9aae-2e415cd247f9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:04.557276 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd40dc-935e-4abd-9aae-2e415cd247f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:04.557749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd40dc-935e-4abd-9aae-2e415cd247f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:04.558689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd40dc-935e-4abd-9aae-2e415cd247f9 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=38462b1165514ed7b948cba710782d4b) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:04.559247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd40dc-935e-4abd-9aae-2e415cd247f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:04.560957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd40dc-935e-4abd-9aae-2e415cd247f9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:04.575862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 372/757] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:04 2022] GET /identity/v3/role_assignments?user.id=38462b1165514ed7b948cba710782d4b&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=4b10782d99b74f5bb453b82286012e0c => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:04.683135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b31291-a68f-417e-bbf7-3276ad0f3769 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:04.709037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:04.724700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dfbcf-87f4-451b-852c-1260922ac75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:04.725349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dfbcf-87f4-451b-852c-1260922ac75a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:04.725626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dfbcf-87f4-451b-852c-1260922ac75a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:04.725897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dfbcf-87f4-451b-852c-1260922ac75a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:04.747722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2dfbcf-87f4-451b-852c-1260922ac75a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:04.749289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2dfbcf-87f4-451b-852c-1260922ac75a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-1b2dfbcf-87f4-451b-852c-1260922ac75a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:04.766686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:04.767718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:04.768605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:04.768605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:04.768605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:04.768605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 386/758] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:04.773764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd40dc-935e-4abd-9aae-2e415cd247f9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:04.882502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:04.888146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd40dc-935e-4abd-9aae-2e415cd247f9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:04.931038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b7543-618d-426a-bc6d-5c8eb12a602c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:04.931675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b7543-618d-426a-bc6d-5c8eb12a602c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:04.931952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b7543-618d-426a-bc6d-5c8eb12a602c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:04.932227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b7543-618d-426a-bc6d-5c8eb12a602c None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:04.932976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b7543-618d-426a-bc6d-5c8eb12a602c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:04.933283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b7543-618d-426a-bc6d-5c8eb12a602c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:04.934739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b7543-618d-426a-bc6d-5c8eb12a602c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:04.949960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 373/759] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:04.955957 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dfbcf-87f4-451b-852c-1260922ac75a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:05.119947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dfbcf-87f4-451b-852c-1260922ac75a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:05.167486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c526bce-552e-4b0d-a8c4-6ea0880ab2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:05.168128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c526bce-552e-4b0d-a8c4-6ea0880ab2b6 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:05.168409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c526bce-552e-4b0d-a8c4-6ea0880ab2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:05.168674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c526bce-552e-4b0d-a8c4-6ea0880ab2b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:05.179589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c526bce-552e-4b0d-a8c4-6ea0880ab2b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:05.181045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c526bce-552e-4b0d-a8c4-6ea0880ab2b6 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-6c526bce-552e-4b0d-a8c4-6ea0880ab2b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:05.199930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:05.201341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:05.202292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 387/760] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 22 17:06:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:05.207999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b7543-618d-426a-bc6d-5c8eb12a602c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:05.317527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b7543-618d-426a-bc6d-5c8eb12a602c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:05.367332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cc7e7-6cd5-42c0-8b37-ba1efcaf736b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:05.367943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cc7e7-6cd5-42c0-8b37-ba1efcaf736b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:05.368215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cc7e7-6cd5-42c0-8b37-ba1efcaf736b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:05.368481 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cc7e7-6cd5-42c0-8b37-ba1efcaf736b None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:05.369218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cc7e7-6cd5-42c0-8b37-ba1efcaf736b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:05.369517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cc7e7-6cd5-42c0-8b37-ba1efcaf736b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:05.370997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cc7e7-6cd5-42c0-8b37-ba1efcaf736b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:05.400853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 374/761] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 22 17:06:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:05.406578 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c526bce-552e-4b0d-a8c4-6ea0880ab2b6 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:05.417997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eedfe-5d00-45a1-91cd-3b0360273476 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:05.418178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eedfe-5d00-45a1-91cd-3b0360273476 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:05.418460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eedfe-5d00-45a1-91cd-3b0360273476 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:05.420342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 375/762] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:05.431500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8609951-3b40-42d7-a6b6-2d44a99effdd None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:05.431798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8609951-3b40-42d7-a6b6-2d44a99effdd None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:05.432046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8609951-3b40-42d7-a6b6-2d44a99effdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:05.458924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a8609951-3b40-42d7-a6b6-2d44a99effdd None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:05.460134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a8609951-3b40-42d7-a6b6-2d44a99effdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:05.461418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:05.478296 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:05.594251 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 376/763] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:05.605357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c526bce-552e-4b0d-a8c4-6ea0880ab2b6 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:05.656917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4435f-20a5-4a5b-93b9-8b4457e68350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:05.657926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4435f-20a5-4a5b-93b9-8b4457e68350 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:05.658384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4435f-20a5-4a5b-93b9-8b4457e68350 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:05.658878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4435f-20a5-4a5b-93b9-8b4457e68350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:05.673521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4435f-20a5-4a5b-93b9-8b4457e68350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:05.675259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4435f-20a5-4a5b-93b9-8b4457e68350 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ecf4435f-20a5-4a5b-93b9-8b4457e68350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:05.696390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:05.697406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:05.698298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:05.698298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:05.698298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:05.700207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 388/764] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:05.706821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8609951-3b40-42d7-a6b6-2d44a99effdd None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:05.816167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8609951-3b40-42d7-a6b6-2d44a99effdd None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:05.848446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b4a10-4869-4ed3-86fb-42091c91ba86 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:05.849021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b4a10-4869-4ed3-86fb-42091c91ba86 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:05.849440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b4a10-4869-4ed3-86fb-42091c91ba86 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:05.851378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 389/765] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:05.860863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c0101-6da5-4d21-8096-39dee2584bc7 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:05.861373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c0101-6da5-4d21-8096-39dee2584bc7 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:05.861876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c0101-6da5-4d21-8096-39dee2584bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:05.870791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3caca-3d01-4578-9a75-06f61159d39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:05.874370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3caca-3d01-4578-9a75-06f61159d39a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:05.874922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3caca-3d01-4578-9a75-06f61159d39a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:05.875340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3caca-3d01-4578-9a75-06f61159d39a None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:05.879119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3caca-3d01-4578-9a75-06f61159d39a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:05.879628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3caca-3d01-4578-9a75-06f61159d39a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:05.881218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3caca-3d01-4578-9a75-06f61159d39a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:05.892909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-243c0101-6da5-4d21-8096-39dee2584bc7 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:05.894306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-243c0101-6da5-4d21-8096-39dee2584bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:05.912412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 377/766] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:05.913904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3caca-3d01-4578-9a75-06f61159d39a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:05.922561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:06.028681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 390/767] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:06.029817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c0101-6da5-4d21-8096-39dee2584bc7 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:06.061410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:06.095932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3caca-3d01-4578-9a75-06f61159d39a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:06.134239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195abe0-1d71-44c7-bc8d-c4514208d940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:06.135325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195abe0-1d71-44c7-bc8d-c4514208d940 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.135802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195abe0-1d71-44c7-bc8d-c4514208d940 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.136272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195abe0-1d71-44c7-bc8d-c4514208d940 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.144894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7195abe0-1d71-44c7-bc8d-c4514208d940 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:06.146692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7195abe0-1d71-44c7-bc8d-c4514208d940 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-7195abe0-1d71-44c7-bc8d-c4514208d940 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:06.159104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:06.160039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:06.161014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:06.161014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:06.161014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:06.161014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:06.161014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:06.161014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:06.161014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:06.161014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:06.163507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 378/768] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:06.170273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195abe0-1d71-44c7-bc8d-c4514208d940 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:06.245207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c0101-6da5-4d21-8096-39dee2584bc7 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:06.287458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:06.311629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6e144-396e-485e-90d7-830b978685fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:06.312612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e144-396e-485e-90d7-830b978685fe None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.312943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e144-396e-485e-90d7-830b978685fe None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.313213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e144-396e-485e-90d7-830b978685fe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.313802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6e144-396e-485e-90d7-830b978685fe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:06.315188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6e144-396e-485e-90d7-830b978685fe None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-d4b6e144-396e-485e-90d7-830b978685fe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:06.324200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:06.325293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:06.326720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 391/769] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Aug 22 17:06:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:06.328350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6e144-396e-485e-90d7-830b978685fe None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:06.336067 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195abe0-1d71-44c7-bc8d-c4514208d940 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:06.374445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f69a57-e6d6-4d8b-a5a6-894db06e04d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:06.375662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69a57-e6d6-4d8b-a5a6-894db06e04d4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.376173 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69a57-e6d6-4d8b-a5a6-894db06e04d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.376624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69a57-e6d6-4d8b-a5a6-894db06e04d4 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.377495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f69a57-e6d6-4d8b-a5a6-894db06e04d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:06.377961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f69a57-e6d6-4d8b-a5a6-894db06e04d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:06.379602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f69a57-e6d6-4d8b-a5a6-894db06e04d4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:06.398098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 379/770] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:06.399346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f69a57-e6d6-4d8b-a5a6-894db06e04d4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:06.423384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:06.439514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6e144-396e-485e-90d7-830b978685fe None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:06.477607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56342a7-9689-4fa3-bed8-55c1dbc92748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:06.478241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56342a7-9689-4fa3-bed8-55c1dbc92748 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.478544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56342a7-9689-4fa3-bed8-55c1dbc92748 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.478817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56342a7-9689-4fa3-bed8-55c1dbc92748 None admin] PATH_INFO: `/v3/services/network` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.479397 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56342a7-9689-4fa3-bed8-55c1dbc92748 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:06.480766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56342a7-9689-4fa3-bed8-55c1dbc92748 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-c56342a7-9689-4fa3-bed8-55c1dbc92748 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:06.492827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:06.494251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:06.495316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 392/771] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:06 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:06.496109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56342a7-9689-4fa3-bed8-55c1dbc92748 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:06.519483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f69a57-e6d6-4d8b-a5a6-894db06e04d4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:06.546414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:06.559495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814e926-efd4-4f35-8b3e-50bb2fbea551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:06.560268 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814e926-efd4-4f35-8b3e-50bb2fbea551 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.560845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814e926-efd4-4f35-8b3e-50bb2fbea551 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.561225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814e926-efd4-4f35-8b3e-50bb2fbea551 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.562064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814e926-efd4-4f35-8b3e-50bb2fbea551 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:06.562607 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814e926-efd4-4f35-8b3e-50bb2fbea551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:06.564102 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814e926-efd4-4f35-8b3e-50bb2fbea551 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:06.574220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 380/772] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 22 17:06:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:06.574220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814e926-efd4-4f35-8b3e-50bb2fbea551 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:06.615764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56342a7-9689-4fa3-bed8-55c1dbc92748 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:06.655357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f0d9f-5f3c-456a-8321-bd0a0e65dbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:06.656648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f0d9f-5f3c-456a-8321-bd0a0e65dbb0 None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.657322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f0d9f-5f3c-456a-8321-bd0a0e65dbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.657757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f0d9f-5f3c-456a-8321-bd0a0e65dbb0 None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/9628118c1df64fa4a7ea8ce37549d8b3/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.659883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f0d9f-5f3c-456a-8321-bd0a0e65dbb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=9628118c1df64fa4a7ea8ce37549d8b3, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:06.661470 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f0d9f-5f3c-456a-8321-bd0a0e65dbb0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:06.691749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814e926-efd4-4f35-8b3e-50bb2fbea551 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:06.730948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6db73-74ec-482b-9f7c-7db8f7d003c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:06.731617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6db73-74ec-482b-9f7c-7db8f7d003c0 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.731886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6db73-74ec-482b-9f7c-7db8f7d003c0 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.732159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6db73-74ec-482b-9f7c-7db8f7d003c0 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.738509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6db73-74ec-482b-9f7c-7db8f7d003c0 None admin] RBAC: Adding query filter params (name=network) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:06.738509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6db73-74ec-482b-9f7c-7db8f7d003c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:06.738509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6db73-74ec-482b-9f7c-7db8f7d003c0 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:06.739229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 393/773] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:06:06 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/9628118c1df64fa4a7ea8ce37549d8b3/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:06:06.740896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f0d9f-5f3c-456a-8321-bd0a0e65dbb0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:06.744094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 381/774] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:06.746514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:06.751782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6db73-74ec-482b-9f7c-7db8f7d003c0 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:06.832547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:06.871418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f0d9f-5f3c-456a-8321-bd0a0e65dbb0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:06.910378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6db73-74ec-482b-9f7c-7db8f7d003c0 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:06.916616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9490720-7649-48a4-8b62-0e7a742e5e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:06.917236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9490720-7649-48a4-8b62-0e7a742e5e45 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.917512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9490720-7649-48a4-8b62-0e7a742e5e45 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.917781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9490720-7649-48a4-8b62-0e7a742e5e45 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.918492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9490720-7649-48a4-8b62-0e7a742e5e45 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:06.918854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9490720-7649-48a4-8b62-0e7a742e5e45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:06.920503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9490720-7649-48a4-8b62-0e7a742e5e45 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:06.927343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 394/775] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 22 17:06:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:06.928361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca903d15-b151-4888-8f05-1732cbe9039b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.928646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca903d15-b151-4888-8f05-1732cbe9039b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.928918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca903d15-b151-4888-8f05-1732cbe9039b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.930360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 395/776] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:06.935105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca903d15-b151-4888-8f05-1732cbe9039b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:06.960628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b207f3-498f-402f-9fe2-52eb515dd953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:06.961624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b207f3-498f-402f-9fe2-52eb515dd953 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.962358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b207f3-498f-402f-9fe2-52eb515dd953 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.962679 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b207f3-498f-402f-9fe2-52eb515dd953 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.963333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b207f3-498f-402f-9fe2-52eb515dd953 None admin] RBAC: Adding query filter params (type=network) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:06.963635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b207f3-498f-402f-9fe2-52eb515dd953 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:06.965556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b207f3-498f-402f-9fe2-52eb515dd953 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:06.973752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 382/777] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:06.974865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2b938-be50-4f14-bf3a-20959af8a5a5 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:06.975153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2b938-be50-4f14-bf3a-20959af8a5a5 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:06.975426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2b938-be50-4f14-bf3a-20959af8a5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:06.995459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2cd2b938-be50-4f14-bf3a-20959af8a5a5 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:06.996658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2cd2b938-be50-4f14-bf3a-20959af8a5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:07.025853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:07.072751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 383/778] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:07.072751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2b938-be50-4f14-bf3a-20959af8a5a5 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:07.073877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca903d15-b151-4888-8f05-1732cbe9039b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:07.116441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179ede6-ef0c-418e-aa3f-22d3c6fe06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:07.117056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ede6-ef0c-418e-aa3f-22d3c6fe06a4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:07.117592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:07.118323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ede6-ef0c-418e-aa3f-22d3c6fe06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:07.118712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ede6-ef0c-418e-aa3f-22d3c6fe06a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:07.119491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179ede6-ef0c-418e-aa3f-22d3c6fe06a4 None admin] RBAC: Adding query filter params (interface=public, service_id=be3a6c0fcabc4027ac750f03bd70e9de, region_id=RegionOne) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:07.119795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179ede6-ef0c-418e-aa3f-22d3c6fe06a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:07.121293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179ede6-ef0c-418e-aa3f-22d3c6fe06a4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:07.129734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 396/779] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 22 17:06:06 2022] GET /identity/v3/endpoints?service_id=be3a6c0fcabc4027ac750f03bd70e9de&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:07.131176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f7264-61d4-4805-ae49-a06fb01834b3 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:07.131543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f7264-61d4-4805-ae49-a06fb01834b3 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:07.131821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f7264-61d4-4805-ae49-a06fb01834b3 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:07.137255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 397/780] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:07.138071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f7264-61d4-4805-ae49-a06fb01834b3 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:07.186675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2b938-be50-4f14-bf3a-20959af8a5a5 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:07.222835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef2e96-21c7-4f2b-b1cd-5f384d8c048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:07.223458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef2e96-21c7-4f2b-b1cd-5f384d8c048e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:07.223736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef2e96-21c7-4f2b-b1cd-5f384d8c048e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:07.224012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef2e96-21c7-4f2b-b1cd-5f384d8c048e None admin] PATH_INFO: `/v3/endpoints` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:07.224723 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef2e96-21c7-4f2b-b1cd-5f384d8c048e None admin] RBAC: Adding query filter params (interface=public, service_id=3ecb8fd618e24a9897d5e897b1a37045, region_id=RegionOne) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:07.225033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef2e96-21c7-4f2b-b1cd-5f384d8c048e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:07.226415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef2e96-21c7-4f2b-b1cd-5f384d8c048e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:07.238603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 384/781] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 22 17:06:07 2022] GET /identity/v3/endpoints?service_id=3ecb8fd618e24a9897d5e897b1a37045&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:07.238603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee3327-4509-4448-9d8f-b50c8700f488 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:07.238603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee3327-4509-4448-9d8f-b50c8700f488 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:07.238603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee3327-4509-4448-9d8f-b50c8700f488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:07.256602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-13ee3327-4509-4448-9d8f-b50c8700f488 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:07.259804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-13ee3327-4509-4448-9d8f-b50c8700f488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:07.261799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f7264-61d4-4805-ae49-a06fb01834b3 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:07.295142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:07.303130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a768122-2bfe-4baa-84a8-1ec2c0f0b6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:07.303948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a768122-2bfe-4baa-84a8-1ec2c0f0b6f7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:07.304395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a768122-2bfe-4baa-84a8-1ec2c0f0b6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:07.304802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a768122-2bfe-4baa-84a8-1ec2c0f0b6f7 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:07.305565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a768122-2bfe-4baa-84a8-1ec2c0f0b6f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:07.307114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a768122-2bfe-4baa-84a8-1ec2c0f0b6f7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:07.312934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 398/782] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 22 17:06:07 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:07.313945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a768122-2bfe-4baa-84a8-1ec2c0f0b6f7 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:07.364258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 385/783] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:07.411409 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee3327-4509-4448-9d8f-b50c8700f488 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:07.432951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a768122-2bfe-4baa-84a8-1ec2c0f0b6f7 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:07.462900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:07.473596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eade61d-8895-4230-b009-3d11b94124ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:07.474535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eade61d-8895-4230-b009-3d11b94124ae None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:07.474960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eade61d-8895-4230-b009-3d11b94124ae None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:07.475374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eade61d-8895-4230-b009-3d11b94124ae None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:07.476144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eade61d-8895-4230-b009-3d11b94124ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:07.477680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eade61d-8895-4230-b009-3d11b94124ae None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:07.483964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 399/784] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 22 17:06:07 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:07.525638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee3327-4509-4448-9d8f-b50c8700f488 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:07.567412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c85502-ad91-46d2-9fe2-e6caf96a28f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:07.568074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c85502-ad91-46d2-9fe2-e6caf96a28f8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:07.568354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c85502-ad91-46d2-9fe2-e6caf96a28f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:07.568635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c85502-ad91-46d2-9fe2-e6caf96a28f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:07.581837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c85502-ad91-46d2-9fe2-e6caf96a28f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-b9c85502-ad91-46d2-9fe2-e6caf96a28f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:07.586706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:07.587707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:07.588553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 386/785] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:06:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:07.594410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eade61d-8895-4230-b009-3d11b94124ae None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:07.664209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:07.684124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c85502-ad91-46d2-9fe2-e6caf96a28f8 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:07.715364 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eade61d-8895-4230-b009-3d11b94124ae None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:07.756212 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123ad4c-72a5-4498-ac80-346998ecf2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:07.756865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123ad4c-72a5-4498-ac80-346998ecf2d5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:07.757154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123ad4c-72a5-4498-ac80-346998ecf2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:07.757424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123ad4c-72a5-4498-ac80-346998ecf2d5 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:07.758156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123ad4c-72a5-4498-ac80-346998ecf2d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:07.758632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123ad4c-72a5-4498-ac80-346998ecf2d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:07.760112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123ad4c-72a5-4498-ac80-346998ecf2d5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:07.769545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 400/786] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:06:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:07.775237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123ad4c-72a5-4498-ac80-346998ecf2d5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:07.854827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:07.857588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c85502-ad91-46d2-9fe2-e6caf96a28f8 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:07.918916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce4698-2f60-4b43-8eac-92d9ca30c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:07.919719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce4698-2f60-4b43-8eac-92d9ca30c326 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:07.920005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce4698-2f60-4b43-8eac-92d9ca30c326 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:07.920277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce4698-2f60-4b43-8eac-92d9ca30c326 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:07.936053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce4698-2f60-4b43-8eac-92d9ca30c326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-8bce4698-2f60-4b43-8eac-92d9ca30c326 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:07.942445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:07.943399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:07.944640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:07.945605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123ad4c-72a5-4498-ac80-346998ecf2d5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:07.945688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 387/787] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:07.955743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce4698-2f60-4b43-8eac-92d9ca30c326 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:07.969953 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:07.992305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ff036-0553-4ce2-ae26-3c1f667576f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:07.992305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ff036-0553-4ce2-ae26-3c1f667576f3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:07.992305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ff036-0553-4ce2-ae26-3c1f667576f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:07.992305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ff036-0553-4ce2-ae26-3c1f667576f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:08.000549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ff036-0553-4ce2-ae26-3c1f667576f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:08.002468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ff036-0553-4ce2-ae26-3c1f667576f3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-da2ff036-0553-4ce2-ae26-3c1f667576f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:08.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:08.022294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:08.024182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:08.024182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:08.024182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:08.024182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 401/788] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:08.032238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ff036-0553-4ce2-ae26-3c1f667576f3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:08.186370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce4698-2f60-4b43-8eac-92d9ca30c326 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:08.190985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ff036-0553-4ce2-ae26-3c1f667576f3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:08.228561 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b2edb-6c74-43e4-a90d-6dc54b77ae26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:08.229171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2edb-6c74-43e4-a90d-6dc54b77ae26 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:08.229451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2edb-6c74-43e4-a90d-6dc54b77ae26 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:08.229767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2edb-6c74-43e4-a90d-6dc54b77ae26 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:08.230413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b2edb-6c74-43e4-a90d-6dc54b77ae26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:08.230743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b2edb-6c74-43e4-a90d-6dc54b77ae26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:08.232117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b2edb-6c74-43e4-a90d-6dc54b77ae26 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:08.245414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 402/789] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:08.251549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b2edb-6c74-43e4-a90d-6dc54b77ae26 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:08.274571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dd4e3-3462-4b4b-96b5-9b46459180b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:08.275194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dd4e3-3462-4b4b-96b5-9b46459180b9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:08.275459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dd4e3-3462-4b4b-96b5-9b46459180b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:08.275733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dd4e3-3462-4b4b-96b5-9b46459180b9 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:08.276406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dd4e3-3462-4b4b-96b5-9b46459180b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:08.276705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dd4e3-3462-4b4b-96b5-9b46459180b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:08.278077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dd4e3-3462-4b4b-96b5-9b46459180b9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:08.285396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:08.304991 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 388/790] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:06:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:08.310231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dd4e3-3462-4b4b-96b5-9b46459180b9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:08.359207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:08.401413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b2edb-6c74-43e4-a90d-6dc54b77ae26 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:08.460662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888e7b3-8f10-4fa9-b919-f8d84abb2b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:08.461221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888e7b3-8f10-4fa9-b919-f8d84abb2b86 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:08.461495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888e7b3-8f10-4fa9-b919-f8d84abb2b86 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:08.461762 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888e7b3-8f10-4fa9-b919-f8d84abb2b86 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:08.474969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888e7b3-8f10-4fa9-b919-f8d84abb2b86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:08.476430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888e7b3-8f10-4fa9-b919-f8d84abb2b86 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-8888e7b3-8f10-4fa9-b919-f8d84abb2b86 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:08.494982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:08.495962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:08.496903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:08.496903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:08.496903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:08.496903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:08.496903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:08.496903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:08.496903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:08.496903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:08.496903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:08.496903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:08.497982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 403/791] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 22 17:06:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:08.500494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dd4e3-3462-4b4b-96b5-9b46459180b9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:08.503700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888e7b3-8f10-4fa9-b919-f8d84abb2b86 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:08.557533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169a569-be86-442b-833f-a01606424d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:08.558164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169a569-be86-442b-833f-a01606424d3b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:08.558439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169a569-be86-442b-833f-a01606424d3b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:08.558739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169a569-be86-442b-833f-a01606424d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:08.581313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169a569-be86-442b-833f-a01606424d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:08.582832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169a569-be86-442b-833f-a01606424d3b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-a169a569-be86-442b-833f-a01606424d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:08.603974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:08.605077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:08.607570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:08.607570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:08.607570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:08.607570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 389/792] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:08.612471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169a569-be86-442b-833f-a01606424d3b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:08.674648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:08.692076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888e7b3-8f10-4fa9-b919-f8d84abb2b86 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:08.753576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b6723-6e50-4bb4-9501-b0958146bbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:08.754127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b6723-6e50-4bb4-9501-b0958146bbec None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:08.754373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b6723-6e50-4bb4-9501-b0958146bbec None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:08.754657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b6723-6e50-4bb4-9501-b0958146bbec None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:08.757178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b6723-6e50-4bb4-9501-b0958146bbec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:08.757491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b6723-6e50-4bb4-9501-b0958146bbec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:08.757910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:08.758966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b6723-6e50-4bb4-9501-b0958146bbec None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:08.794770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 404/793] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 22 17:06:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:08.802224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b6723-6e50-4bb4-9501-b0958146bbec None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:08.803213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169a569-be86-442b-833f-a01606424d3b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:08.878317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40838a9-a7e5-4e24-a0a2-0821cb4cc648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:08.878986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40838a9-a7e5-4e24-a0a2-0821cb4cc648 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:08.879263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40838a9-a7e5-4e24-a0a2-0821cb4cc648 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:08.879536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40838a9-a7e5-4e24-a0a2-0821cb4cc648 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:08.880313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40838a9-a7e5-4e24-a0a2-0821cb4cc648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:08.880619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40838a9-a7e5-4e24-a0a2-0821cb4cc648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:08.882275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40838a9-a7e5-4e24-a0a2-0821cb4cc648 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:08.910967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 390/794] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:08.916345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40838a9-a7e5-4e24-a0a2-0821cb4cc648 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:08.996653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b6723-6e50-4bb4-9501-b0958146bbec None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:09.057804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40838a9-a7e5-4e24-a0a2-0821cb4cc648 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:09.059366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4369be-d4a1-4a4c-a7fc-a584bdebabb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:09.060048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4369be-d4a1-4a4c-a7fc-a584bdebabb2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.060322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4369be-d4a1-4a4c-a7fc-a584bdebabb2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.060588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4369be-d4a1-4a4c-a7fc-a584bdebabb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.077285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4369be-d4a1-4a4c-a7fc-a584bdebabb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:09.079053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4369be-d4a1-4a4c-a7fc-a584bdebabb2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ad4369be-d4a1-4a4c-a7fc-a584bdebabb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:09.097050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:09.098137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:09.099741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:09.099741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:09.099741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:09.102579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 405/795] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:09.103973 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e686dec-6019-477a-a9c2-93c0f07b61e4 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.104254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e686dec-6019-477a-a9c2-93c0f07b61e4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.104579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e686dec-6019-477a-a9c2-93c0f07b61e4 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.110819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 406/796] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:09.110819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e686dec-6019-477a-a9c2-93c0f07b61e4 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:09.111642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:09.123610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b1e38-d7b3-4e1b-8365-6c9725155004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:09.129064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b1e38-d7b3-4e1b-8365-6c9725155004 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.129064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b1e38-d7b3-4e1b-8365-6c9725155004 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.129064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b1e38-d7b3-4e1b-8365-6c9725155004 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.129064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:09.136146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b1e38-d7b3-4e1b-8365-6c9725155004 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:09.138668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b1e38-d7b3-4e1b-8365-6c9725155004 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-558b1e38-d7b3-4e1b-8365-6c9725155004 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:09.153843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:09.154923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:09.156574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:09.156574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:09.156574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:09.156574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:09.156574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:09.156574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:09.156574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:09.156574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:09.156574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:06:09.156574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:09.160050 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 391/797] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:09.161896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475d470-e2a2-4400-9ef6-21376f632050 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.162588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475d470-e2a2-4400-9ef6-21376f632050 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.163471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475d470-e2a2-4400-9ef6-21376f632050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.189024 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-9475d470-e2a2-4400-9ef6-21376f632050 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:09.191076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9475d470-e2a2-4400-9ef6-21376f632050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:09.221532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e686dec-6019-477a-a9c2-93c0f07b61e4 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:09.274507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 392/798] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:09.274507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475d470-e2a2-4400-9ef6-21376f632050 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:09.277012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafa7ee-5909-499c-955d-709d4b2a4fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:09.277634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa7ee-5909-499c-955d-709d4b2a4fd4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.277906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa7ee-5909-499c-955d-709d4b2a4fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.280636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa7ee-5909-499c-955d-709d4b2a4fd4 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.281489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafa7ee-5909-499c-955d-709d4b2a4fd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:09.281884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafa7ee-5909-499c-955d-709d4b2a4fd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:09.285034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafa7ee-5909-499c-955d-709d4b2a4fd4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:09.311766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 407/799] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:09.313261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafa7ee-5909-499c-955d-709d4b2a4fd4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:09.416742 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:09.452220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafa7ee-5909-499c-955d-709d4b2a4fd4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:09.459678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475d470-e2a2-4400-9ef6-21376f632050 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:09.506844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:09.507833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614ce4f-f76e-402b-bcf3-38da7e3db503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:09.508486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614ce4f-f76e-402b-bcf3-38da7e3db503 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.508758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614ce4f-f76e-402b-bcf3-38da7e3db503 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.509020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614ce4f-f76e-402b-bcf3-38da7e3db503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.523186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614ce4f-f76e-402b-bcf3-38da7e3db503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:09.525205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d175955-3218-403d-aa91-229ae917af19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:09.525869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d175955-3218-403d-aa91-229ae917af19 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.526142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d175955-3218-403d-aa91-229ae917af19 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.526404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d175955-3218-403d-aa91-229ae917af19 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.527172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d175955-3218-403d-aa91-229ae917af19 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:09.527472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d175955-3218-403d-aa91-229ae917af19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:09.528945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d175955-3218-403d-aa91-229ae917af19 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:09.530834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614ce4f-f76e-402b-bcf3-38da7e3db503 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-4614ce4f-f76e-402b-bcf3-38da7e3db503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:09.556071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:09.557111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:09.558034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:09.558034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:09.558034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:09.558034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:09.558034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:09.558034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:09.558034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:09.558034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:09.565012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 408/800] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:09.571144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614ce4f-f76e-402b-bcf3-38da7e3db503 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:09.576056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 393/801] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 22 17:06:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:09.581602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d175955-3218-403d-aa91-229ae917af19 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:09.708767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d175955-3218-403d-aa91-229ae917af19 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:09.743612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614ce4f-f76e-402b-bcf3-38da7e3db503 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:09.745189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb47601-cb07-4595-ae10-eee3d8f9ee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:09.745818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb47601-cb07-4595-ae10-eee3d8f9ee55 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.746092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb47601-cb07-4595-ae10-eee3d8f9ee55 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.746357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb47601-cb07-4595-ae10-eee3d8f9ee55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.766174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb47601-cb07-4595-ae10-eee3d8f9ee55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:09.767687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb47601-cb07-4595-ae10-eee3d8f9ee55 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:09.769826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:09.782589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447770f-f45f-4152-8fe3-c9990df335d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-5cb47601-cb07-4595-ae10-eee3d8f9ee55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:09.783426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:09.784656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:09.786311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:09.786311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:09.786311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:09.786516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447770f-f45f-4152-8fe3-c9990df335d7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.786516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447770f-f45f-4152-8fe3-c9990df335d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.786516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447770f-f45f-4152-8fe3-c9990df335d7 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.786749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 394/802] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:09.786749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85a167-9b6e-4df8-a8b3-0445c0ceb6ae None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.786749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85a167-9b6e-4df8-a8b3-0445c0ceb6ae None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.786749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85a167-9b6e-4df8-a8b3-0445c0ceb6ae None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.790622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 395/803] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:09.790622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85a167-9b6e-4df8-a8b3-0445c0ceb6ae None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:09.790832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447770f-f45f-4152-8fe3-c9990df335d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:09.790832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447770f-f45f-4152-8fe3-c9990df335d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:09.791425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447770f-f45f-4152-8fe3-c9990df335d7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:09.805978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 409/804] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:09.807010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447770f-f45f-4152-8fe3-c9990df335d7 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:09.827560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:09.896383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85a167-9b6e-4df8-a8b3-0445c0ceb6ae None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:09.912242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447770f-f45f-4152-8fe3-c9990df335d7 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:09.931680 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34d13f-4ef7-43e9-a531-a90a876331b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:09.932305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d13f-4ef7-43e9-a531-a90a876331b5 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.932578 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d13f-4ef7-43e9-a531-a90a876331b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.932846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d13f-4ef7-43e9-a531-a90a876331b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.943197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34d13f-4ef7-43e9-a531-a90a876331b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:09.945398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c9008-3242-4450-96b2-02de6f2dc680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:09.946000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c9008-3242-4450-96b2-02de6f2dc680 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.946278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c9008-3242-4450-96b2-02de6f2dc680 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-5f34d13f-4ef7-43e9-a531-a90a876331b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:09.947822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:09.948802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:09.949671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:09.950327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c9008-3242-4450-96b2-02de6f2dc680 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.950399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 396/805] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:09.950473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c9008-3242-4450-96b2-02de6f2dc680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:09.950473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c9008-3242-4450-96b2-02de6f2dc680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:09.950646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1f2fe-0208-4744-b7d2-be0c3456118b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.950646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1f2fe-0208-4744-b7d2-be0c3456118b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.950782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1f2fe-0208-4744-b7d2-be0c3456118b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.950855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c9008-3242-4450-96b2-02de6f2dc680 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:09.963082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 410/806] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:09.964374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0658a5-48a5-4b4a-9bb6-5a5d9fe8f41c None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:09.964662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0658a5-48a5-4b4a-9bb6-5a5d9fe8f41c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:09.964933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0658a5-48a5-4b4a-9bb6-5a5d9fe8f41c None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:09.966255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 411/807] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:09.966879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0658a5-48a5-4b4a-9bb6-5a5d9fe8f41c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:09.973625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-87c1f2fe-0208-4744-b7d2-be0c3456118b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:09.974960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-87c1f2fe-0208-4744-b7d2-be0c3456118b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:10.011118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:10.042147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 397/808] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:10.043104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1f2fe-0208-4744-b7d2-be0c3456118b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:10.070914 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0658a5-48a5-4b4a-9bb6-5a5d9fe8f41c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:10.076841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:10.108419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a9466-5a0a-4fd8-b6a9-a420536ae38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:10.109058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a9466-5a0a-4fd8-b6a9-a420536ae38f None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:10.109393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a9466-5a0a-4fd8-b6a9-a420536ae38f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:10.109670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a9466-5a0a-4fd8-b6a9-a420536ae38f None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/9001ee719e294f79b35f13389ac9cbaa/roles/14a34ac79fe64a978eca3e5f9089e72f` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:10.124368 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a9466-5a0a-4fd8-b6a9-a420536ae38f None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=9001ee719e294f79b35f13389ac9cbaa, role_id=14a34ac79fe64a978eca3e5f9089e72f)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:10.125820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a9466-5a0a-4fd8-b6a9-a420536ae38f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:10.158394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1f2fe-0208-4744-b7d2-be0c3456118b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:10.184883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 412/809] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:06:09 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/9001ee719e294f79b35f13389ac9cbaa/roles/14a34ac79fe64a978eca3e5f9089e72f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:06:10.188338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a9466-5a0a-4fd8-b6a9-a420536ae38f None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:10.205834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4d053-e6df-47cc-8d2a-87dff70b4fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:10.206739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4d053-e6df-47cc-8d2a-87dff70b4fd8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:10.207087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4d053-e6df-47cc-8d2a-87dff70b4fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:10.207497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4d053-e6df-47cc-8d2a-87dff70b4fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:10.208364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4d053-e6df-47cc-8d2a-87dff70b4fd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:10.208848 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4d053-e6df-47cc-8d2a-87dff70b4fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:10.210406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4d053-e6df-47cc-8d2a-87dff70b4fd8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:10.221903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 398/810] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:06:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:10.223406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cdb19-204b-4bc5-9dc8-668b24a55548 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:10.223697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cdb19-204b-4bc5-9dc8-668b24a55548 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:10.224053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cdb19-204b-4bc5-9dc8-668b24a55548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:10.252824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-bc9cdb19-204b-4bc5-9dc8-668b24a55548 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:10.254871 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bc9cdb19-204b-4bc5-9dc8-668b24a55548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:10.339602 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:10.352114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a9466-5a0a-4fd8-b6a9-a420536ae38f None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:10.356733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 399/811] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:10.357881 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cdb19-204b-4bc5-9dc8-668b24a55548 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:10.382053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:10.399001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684a8bb-78e0-4747-8d40-87d85a9e834a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:10.399936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684a8bb-78e0-4747-8d40-87d85a9e834a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:10.400110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684a8bb-78e0-4747-8d40-87d85a9e834a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:10.400496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684a8bb-78e0-4747-8d40-87d85a9e834a None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:10.410559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684a8bb-78e0-4747-8d40-87d85a9e834a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:10.412030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684a8bb-78e0-4747-8d40-87d85a9e834a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-5684a8bb-78e0-4747-8d40-87d85a9e834a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:10.427311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:10.428274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:10.429168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:10.429168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:10.429168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:10.429168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:10.429168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:10.429168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:10.429168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:10.429168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:10.429168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 413/812] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:10.430126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684a8bb-78e0-4747-8d40-87d85a9e834a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:10.507935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cdb19-204b-4bc5-9dc8-668b24a55548 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:10.549355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684a8bb-78e0-4747-8d40-87d85a9e834a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:10.556439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9b275-d437-424f-9b78-83017ee4f1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:10.557133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9b275-d437-424f-9b78-83017ee4f1be None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:10.557423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9b275-d437-424f-9b78-83017ee4f1be None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:10.557697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9b275-d437-424f-9b78-83017ee4f1be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:10.573035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9b275-d437-424f-9b78-83017ee4f1be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:10.574819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9b275-d437-424f-9b78-83017ee4f1be None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:10.590833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57677ede-9a10-4098-a619-5e16fec33daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:10.591469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57677ede-9a10-4098-a619-5e16fec33daf None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:10.591745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57677ede-9a10-4098-a619-5e16fec33daf None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:10.592015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57677ede-9a10-4098-a619-5e16fec33daf None admin] PATH_INFO: `/v3/services/network` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:10.592601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57677ede-9a10-4098-a619-5e16fec33daf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:10.594521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57677ede-9a10-4098-a619-5e16fec33daf None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-92d9b275-d437-424f-9b78-83017ee4f1be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:10.595089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:10.596891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:10.598693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:10.598693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:10.598693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:10.598693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 400/813] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:10.598693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9b275-d437-424f-9b78-83017ee4f1be None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-57677ede-9a10-4098-a619-5e16fec33daf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:10.607259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:10.608836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 17:06:10.611193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:10.618536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 414/814] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:10 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:10.618536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57677ede-9a10-4098-a619-5e16fec33daf None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:10.641069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:10.686099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:10.726744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9b275-d437-424f-9b78-83017ee4f1be None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:10.728025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57677ede-9a10-4098-a619-5e16fec33daf None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:10.762902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ff15a-477f-40d3-a53e-d69d3fb3e500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:10.763513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ff15a-477f-40d3-a53e-d69d3fb3e500 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:10.763794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ff15a-477f-40d3-a53e-d69d3fb3e500 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:10.764068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ff15a-477f-40d3-a53e-d69d3fb3e500 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:10.764763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ff15a-477f-40d3-a53e-d69d3fb3e500 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:10.765186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ff15a-477f-40d3-a53e-d69d3fb3e500 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:10.766628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ff15a-477f-40d3-a53e-d69d3fb3e500 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:10.766749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29213022-2cfe-4063-bb44-fbc883be3ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:10.767308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29213022-2cfe-4063-bb44-fbc883be3ad7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:10.767584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29213022-2cfe-4063-bb44-fbc883be3ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:10.767854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29213022-2cfe-4063-bb44-fbc883be3ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:10.769031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29213022-2cfe-4063-bb44-fbc883be3ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:10.769332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29213022-2cfe-4063-bb44-fbc883be3ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:10.770888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29213022-2cfe-4063-bb44-fbc883be3ad7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:10.785317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 401/815] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:10.785317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ff15a-477f-40d3-a53e-d69d3fb3e500 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:10.793456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 415/816] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:10.799884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29213022-2cfe-4063-bb44-fbc883be3ad7 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:10.883495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:10.899120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ff15a-477f-40d3-a53e-d69d3fb3e500 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:10.906046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29213022-2cfe-4063-bb44-fbc883be3ad7 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:10.929729 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:10.938919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129f035-a39e-4545-8386-377d32d2efc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:10.942628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129f035-a39e-4545-8386-377d32d2efc0 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:10.942628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129f035-a39e-4545-8386-377d32d2efc0 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:10.942628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129f035-a39e-4545-8386-377d32d2efc0 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:10.942628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129f035-a39e-4545-8386-377d32d2efc0 None admin] RBAC: Adding query filter params (name=network) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:10.942628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129f035-a39e-4545-8386-377d32d2efc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:10.942628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129f035-a39e-4545-8386-377d32d2efc0 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:10.944379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8948-30e1-4632-9494-939d089f99f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:10.945014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8948-30e1-4632-9494-939d089f99f7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:10.945296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8948-30e1-4632-9494-939d089f99f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:10.945572 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8948-30e1-4632-9494-939d089f99f7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:10.946156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8948-30e1-4632-9494-939d089f99f7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:10.951758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 402/817] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:10.951894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8948-30e1-4632-9494-939d089f99f7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:10.952008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129f035-a39e-4545-8386-377d32d2efc0 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-2fbf8948-30e1-4632-9494-939d089f99f7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:10.960909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:10.961942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 17:06:10.962851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:10.966786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 416/818] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Aug 22 17:06:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:10.967501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8948-30e1-4632-9494-939d089f99f7 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:11.065601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129f035-a39e-4545-8386-377d32d2efc0 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:11.074161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8948-30e1-4632-9494-939d089f99f7 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:11.103493 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089f2e4-52cb-49b7-8cf1-2266194ff893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:11.103763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089f2e4-52cb-49b7-8cf1-2266194ff893 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:11.105040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089f2e4-52cb-49b7-8cf1-2266194ff893 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:11.105040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089f2e4-52cb-49b7-8cf1-2266194ff893 None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/38462b1165514ed7b948cba710782d4b/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:11.106396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089f2e4-52cb-49b7-8cf1-2266194ff893 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=38462b1165514ed7b948cba710782d4b, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:11.107882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089f2e4-52cb-49b7-8cf1-2266194ff893 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:11.117729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b11cb1-24de-4a33-990f-6a7b4ea7d873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:11.118457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b11cb1-24de-4a33-990f-6a7b4ea7d873 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:11.118830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b11cb1-24de-4a33-990f-6a7b4ea7d873 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:11.119797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b11cb1-24de-4a33-990f-6a7b4ea7d873 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:11.128592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b11cb1-24de-4a33-990f-6a7b4ea7d873 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:11.130041 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b11cb1-24de-4a33-990f-6a7b4ea7d873 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:11.132744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-47b11cb1-24de-4a33-990f-6a7b4ea7d873 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:11.142475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:11.143510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 417/819] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 22 17:06:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:11.144456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b11cb1-24de-4a33-990f-6a7b4ea7d873 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:11.197328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 403/820] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:06:10 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/38462b1165514ed7b948cba710782d4b/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:06:11.198530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089f2e4-52cb-49b7-8cf1-2266194ff893 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:11.272553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b11cb1-24de-4a33-990f-6a7b4ea7d873 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:11.276433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:11.315625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074068a2-44d6-4e76-9d1a-ca53fc9ad821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:11.316362 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074068a2-44d6-4e76-9d1a-ca53fc9ad821 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:11.316819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074068a2-44d6-4e76-9d1a-ca53fc9ad821 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:11.317072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074068a2-44d6-4e76-9d1a-ca53fc9ad821 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:11.317802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074068a2-44d6-4e76-9d1a-ca53fc9ad821 None admin] RBAC: Adding query filter params (type=network) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:11.318178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074068a2-44d6-4e76-9d1a-ca53fc9ad821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:11.319731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074068a2-44d6-4e76-9d1a-ca53fc9ad821 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:11.330519 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 418/821] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:11.330519 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074068a2-44d6-4e76-9d1a-ca53fc9ad821 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:11.345166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089f2e4-52cb-49b7-8cf1-2266194ff893 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:11.391398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf7898-a798-45ad-aaba-72d9bd2e8dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:11.392333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf7898-a798-45ad-aaba-72d9bd2e8dae None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:11.392806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf7898-a798-45ad-aaba-72d9bd2e8dae None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:11.393073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf7898-a798-45ad-aaba-72d9bd2e8dae None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:11.393713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf7898-a798-45ad-aaba-72d9bd2e8dae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:11.394012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf7898-a798-45ad-aaba-72d9bd2e8dae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:11.396185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf7898-a798-45ad-aaba-72d9bd2e8dae None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:11.401733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 404/822] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 22 17:06:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:11.402352 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf7898-a798-45ad-aaba-72d9bd2e8dae None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:11.403820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:11.452318 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074068a2-44d6-4e76-9d1a-ca53fc9ad821 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:11.500485 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e176141-d2a5-444d-9ac3-881897d00ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:11.501105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e176141-d2a5-444d-9ac3-881897d00ae8 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:11.501438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e176141-d2a5-444d-9ac3-881897d00ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:11.501710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e176141-d2a5-444d-9ac3-881897d00ae8 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:11.502636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e176141-d2a5-444d-9ac3-881897d00ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:11.502945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e176141-d2a5-444d-9ac3-881897d00ae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:11.504542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e176141-d2a5-444d-9ac3-881897d00ae8 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:11.523734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf7898-a798-45ad-aaba-72d9bd2e8dae None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:11.526220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 419/823] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 22 17:06:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:11.527183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e176141-d2a5-444d-9ac3-881897d00ae8 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:11.550299 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:11.570142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce700cf-4a49-40d6-bdce-51a6072fa1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:11.571087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce700cf-4a49-40d6-bdce-51a6072fa1ce None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:11.571546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce700cf-4a49-40d6-bdce-51a6072fa1ce None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:11.572023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce700cf-4a49-40d6-bdce-51a6072fa1ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:11.572833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce700cf-4a49-40d6-bdce-51a6072fa1ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:11.574453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce700cf-4a49-40d6-bdce-51a6072fa1ce None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:11.643377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e176141-d2a5-444d-9ac3-881897d00ae8 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:11.669195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:11.681419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d14c9d-91e5-4b36-a299-04b96089c2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:11.682098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d14c9d-91e5-4b36-a299-04b96089c2d0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:11.682489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d14c9d-91e5-4b36-a299-04b96089c2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:11.682850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d14c9d-91e5-4b36-a299-04b96089c2d0 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:11.683566 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d14c9d-91e5-4b36-a299-04b96089c2d0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:11.683875 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d14c9d-91e5-4b36-a299-04b96089c2d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:11.685310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d14c9d-91e5-4b36-a299-04b96089c2d0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:11.694417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 405/824] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:06:11 2022] POST /identity/v3/endpoints => generated 337 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:11.694417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce700cf-4a49-40d6-bdce-51a6072fa1ce None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:11.697892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 420/825] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 22 17:06:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:11.706006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d14c9d-91e5-4b36-a299-04b96089c2d0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:11.865189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce700cf-4a49-40d6-bdce-51a6072fa1ce None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:11.897435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d14c9d-91e5-4b36-a299-04b96089c2d0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:11.923981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb885735-b35f-44a2-be97-12d824852269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:11.926158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb885735-b35f-44a2-be97-12d824852269 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:11.926713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb885735-b35f-44a2-be97-12d824852269 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:11.927178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb885735-b35f-44a2-be97-12d824852269 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:11.942778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7731f2-4f19-44f6-b6b6-2513635a532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:11.943535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7731f2-4f19-44f6-b6b6-2513635a532a None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:11.943816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7731f2-4f19-44f6-b6b6-2513635a532a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:11.944091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7731f2-4f19-44f6-b6b6-2513635a532a None admin] PATH_INFO: `/v3/endpoints` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:11.944769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7731f2-4f19-44f6-b6b6-2513635a532a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:11.947559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7731f2-4f19-44f6-b6b6-2513635a532a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:11.975019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:12.013133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb885735-b35f-44a2-be97-12d824852269 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:12.014724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb885735-b35f-44a2-be97-12d824852269 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:12.039381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 421/826] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:06:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:06:12.134181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 406/827] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:06:12.139841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7731f2-4f19-44f6-b6b6-2513635a532a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:12.189816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:12.260989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7731f2-4f19-44f6-b6b6-2513635a532a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:12.297632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf5dab-8c54-4fc4-91c8-c7786f5b6d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:12.298276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf5dab-8c54-4fc4-91c8-c7786f5b6d9c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:12.298586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf5dab-8c54-4fc4-91c8-c7786f5b6d9c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:12.298864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf5dab-8c54-4fc4-91c8-c7786f5b6d9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:12.310514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf5dab-8c54-4fc4-91c8-c7786f5b6d9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:12.312027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf5dab-8c54-4fc4-91c8-c7786f5b6d9c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:12.315816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74659ad-aa5f-4a8d-a628-083422d52abb None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:12.316283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74659ad-aa5f-4a8d-a628-083422d52abb None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:12.316360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74659ad-aa5f-4a8d-a628-083422d52abb None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:12.317918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 407/828] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:12.323191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f388-1e24-47ac-a3f0-950bb7ab894d None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:12.323467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f388-1e24-47ac-a3f0-950bb7ab894d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:12.323741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f388-1e24-47ac-a3f0-950bb7ab894d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-23cf5dab-8c54-4fc4-91c8-c7786f5b6d9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:12.328050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:12.329912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:12.331207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:12.331207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:12.331207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:12.331207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 422/829] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:12.334995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf5dab-8c54-4fc4-91c8-c7786f5b6d9c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:12.343820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-7b39f388-1e24-47ac-a3f0-950bb7ab894d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:12.345117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7b39f388-1e24-47ac-a3f0-950bb7ab894d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:12.436866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:12.442514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:12.450572 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 408/830] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:12.461387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf5dab-8c54-4fc4-91c8-c7786f5b6d9c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:12.501277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d0812-2e44-4766-9b1e-62cdbf252913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:12.502264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d0812-2e44-4766-9b1e-62cdbf252913 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:12.502789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d0812-2e44-4766-9b1e-62cdbf252913 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:12.503317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d0812-2e44-4766-9b1e-62cdbf252913 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:12.504219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d0812-2e44-4766-9b1e-62cdbf252913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:12.504716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d0812-2e44-4766-9b1e-62cdbf252913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:12.506357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d0812-2e44-4766-9b1e-62cdbf252913 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:12.520319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 423/831] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:12.527584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39f388-1e24-47ac-a3f0-950bb7ab894d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:12.634713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39f388-1e24-47ac-a3f0-950bb7ab894d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:12.673277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0585a46-5011-40d4-87df-9e20c2b0749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:12.673917 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0585a46-5011-40d4-87df-9e20c2b0749b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:12.674199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0585a46-5011-40d4-87df-9e20c2b0749b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:12.674472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0585a46-5011-40d4-87df-9e20c2b0749b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:12.681735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0585a46-5011-40d4-87df-9e20c2b0749b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:12.683283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0585a46-5011-40d4-87df-9e20c2b0749b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-f0585a46-5011-40d4-87df-9e20c2b0749b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:12.696801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:12.697748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:12.699342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:12.699342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:12.699342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:12.699342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:12.699342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:12.699342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:12.699342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:12.699342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:12.699342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 409/832] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:12.704458 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d0812-2e44-4766-9b1e-62cdbf252913 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:12.764877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0585a46-5011-40d4-87df-9e20c2b0749b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:12.818534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d0812-2e44-4766-9b1e-62cdbf252913 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:12.829577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:12.856724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666255f-a772-4eae-bbe1-ec4b31e5af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:12.857498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666255f-a772-4eae-bbe1-ec4b31e5af09 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:12.857873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666255f-a772-4eae-bbe1-ec4b31e5af09 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:12.858314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666255f-a772-4eae-bbe1-ec4b31e5af09 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:12.859647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666255f-a772-4eae-bbe1-ec4b31e5af09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:12.860241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666255f-a772-4eae-bbe1-ec4b31e5af09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:12.862687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666255f-a772-4eae-bbe1-ec4b31e5af09 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:12.880746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 424/833] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:12.880926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0585a46-5011-40d4-87df-9e20c2b0749b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:12.882171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666255f-a772-4eae-bbe1-ec4b31e5af09 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:12.897263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:12.925700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd00c21-50b0-4733-9f76-b3f3a5555c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:12.926330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00c21-50b0-4733-9f76-b3f3a5555c0e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:12.926639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00c21-50b0-4733-9f76-b3f3a5555c0e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:12.926908 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00c21-50b0-4733-9f76-b3f3a5555c0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:12.941524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd00c21-50b0-4733-9f76-b3f3a5555c0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-8cd00c21-50b0-4733-9f76-b3f3a5555c0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:12.948889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:12.949904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:06:12.950798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:12.954563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 410/834] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:06:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:12.956240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd00c21-50b0-4733-9f76-b3f3a5555c0e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:13.024130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666255f-a772-4eae-bbe1-ec4b31e5af09 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:13.070362 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a9906-cadf-4d3b-a5d5-182e011a23a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:13.071859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a9906-cadf-4d3b-a5d5-182e011a23a6 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:13.072285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a9906-cadf-4d3b-a5d5-182e011a23a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:13.072943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a9906-cadf-4d3b-a5d5-182e011a23a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:13.073807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a9906-cadf-4d3b-a5d5-182e011a23a6 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=9628118c1df64fa4a7ea8ce37549d8b3) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:13.074534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a9906-cadf-4d3b-a5d5-182e011a23a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:13.076818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a9906-cadf-4d3b-a5d5-182e011a23a6 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:13.082411 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd00c21-50b0-4733-9f76-b3f3a5555c0e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:13.085487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 425/835] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:12 2022] GET /identity/v3/role_assignments?user.id=9628118c1df64fa4a7ea8ce37549d8b3&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=4b10782d99b74f5bb453b82286012e0c => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:13.104514 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:13.126807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3651e9-ec98-4f30-88c6-624bb16eae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:13.127758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3651e9-ec98-4f30-88c6-624bb16eae89 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:13.128238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3651e9-ec98-4f30-88c6-624bb16eae89 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:13.128652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3651e9-ec98-4f30-88c6-624bb16eae89 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:13.129486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3651e9-ec98-4f30-88c6-624bb16eae89 None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:13.129954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3651e9-ec98-4f30-88c6-624bb16eae89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:13.131555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3651e9-ec98-4f30-88c6-624bb16eae89 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:13.143005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 411/836] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:06:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:13.149021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a9906-cadf-4d3b-a5d5-182e011a23a6 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:13.235098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:13.267592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a9906-cadf-4d3b-a5d5-182e011a23a6 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:13.276560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552569a-930c-4b0c-b30d-088006cbd321 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:13.276854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552569a-930c-4b0c-b30d-088006cbd321 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:13.277101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552569a-930c-4b0c-b30d-088006cbd321 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:13.278428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 412/837] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:13.284005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409836f4-42f0-4779-9a96-8a1a727ca218 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:13.284384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409836f4-42f0-4779-9a96-8a1a727ca218 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:13.284634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409836f4-42f0-4779-9a96-8a1a727ca218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:13.305546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-409836f4-42f0-4779-9a96-8a1a727ca218 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:13.307438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-409836f4-42f0-4779-9a96-8a1a727ca218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:13.315102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a77ea-6ab9-4e96-b5bc-003a5dbe6378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:13.316072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a77ea-6ab9-4e96-b5bc-003a5dbe6378 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:13.316480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a77ea-6ab9-4e96-b5bc-003a5dbe6378 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:13.317459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a77ea-6ab9-4e96-b5bc-003a5dbe6378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:13.333271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a77ea-6ab9-4e96-b5bc-003a5dbe6378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:13.334678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a77ea-6ab9-4e96-b5bc-003a5dbe6378 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-7a8a77ea-6ab9-4e96-b5bc-003a5dbe6378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:13.359501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:13.360545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:13.361466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:13.361466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:13.361466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:13.361466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 426/838] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:13.367511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a77ea-6ab9-4e96-b5bc-003a5dbe6378 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:13.402131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 413/839] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:13.490098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a77ea-6ab9-4e96-b5bc-003a5dbe6378 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:13.517204 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:13.526045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85263eb6-08e4-43e5-aa7b-a968552b29a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:13.526670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263eb6-08e4-43e5-aa7b-a968552b29a6 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:13.532115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263eb6-08e4-43e5-aa7b-a968552b29a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:13.532115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263eb6-08e4-43e5-aa7b-a968552b29a6 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:13.532115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85263eb6-08e4-43e5-aa7b-a968552b29a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:13.532115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85263eb6-08e4-43e5-aa7b-a968552b29a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:13.532115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85263eb6-08e4-43e5-aa7b-a968552b29a6 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:13.543672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 427/840] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:13.550164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409836f4-42f0-4779-9a96-8a1a727ca218 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:13.655169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409836f4-42f0-4779-9a96-8a1a727ca218 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:13.670133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:13.692158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21978d7c-7250-4b9e-bbb4-4559fb8bf138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:13.692880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21978d7c-7250-4b9e-bbb4-4559fb8bf138 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:13.693106 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21978d7c-7250-4b9e-bbb4-4559fb8bf138 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:13.693379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21978d7c-7250-4b9e-bbb4-4559fb8bf138 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:13.694648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85263eb6-08e4-43e5-aa7b-a968552b29a6 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:13.702171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21978d7c-7250-4b9e-bbb4-4559fb8bf138 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:13.703826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21978d7c-7250-4b9e-bbb4-4559fb8bf138 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-21978d7c-7250-4b9e-bbb4-4559fb8bf138 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:13.719576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:13.720625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:13.721584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 414/841] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 22 17:06:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:13.726900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21978d7c-7250-4b9e-bbb4-4559fb8bf138 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:13.818330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85263eb6-08e4-43e5-aa7b-a968552b29a6 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:13.849297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21978d7c-7250-4b9e-bbb4-4559fb8bf138 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:13.863160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d61cbb-d9c6-4e0f-b214-43a2f88f5d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:13.864278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d61cbb-d9c6-4e0f-b214-43a2f88f5d84 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:13.864820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d61cbb-d9c6-4e0f-b214-43a2f88f5d84 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:13.865328 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d61cbb-d9c6-4e0f-b214-43a2f88f5d84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:13.877169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d61cbb-d9c6-4e0f-b214-43a2f88f5d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-94d61cbb-d9c6-4e0f-b214-43a2f88f5d84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:13.882694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:13.883639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:13.884545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:13.887376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 428/842] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:13.892496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48a9dd-0b54-4ff9-847d-c68aef72e7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:13.893319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48a9dd-0b54-4ff9-847d-c68aef72e7c2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:13.893425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d61cbb-d9c6-4e0f-b214-43a2f88f5d84 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:13.893945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48a9dd-0b54-4ff9-847d-c68aef72e7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:13.894415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48a9dd-0b54-4ff9-847d-c68aef72e7c2 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:13.895364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48a9dd-0b54-4ff9-847d-c68aef72e7c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:13.895842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48a9dd-0b54-4ff9-847d-c68aef72e7c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:13.897795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48a9dd-0b54-4ff9-847d-c68aef72e7c2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:13.926643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 415/843] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 22 17:06:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:13.927261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48a9dd-0b54-4ff9-847d-c68aef72e7c2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:13.953168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:13.964561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:14.025500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d61cbb-d9c6-4e0f-b214-43a2f88f5d84 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:14.044649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48a9dd-0b54-4ff9-847d-c68aef72e7c2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:14.073840 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3300a-d57b-44a0-9422-30cb90d549bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:14.074860 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3300a-d57b-44a0-9422-30cb90d549bb None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.075279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3300a-d57b-44a0-9422-30cb90d549bb None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.075906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3300a-d57b-44a0-9422-30cb90d549bb None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.077179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3300a-d57b-44a0-9422-30cb90d549bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:14.078061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3300a-d57b-44a0-9422-30cb90d549bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:14.080336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3300a-d57b-44a0-9422-30cb90d549bb None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:14.085009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818d585-e3ad-4a46-a462-2e7b7fc795c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:14.085734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818d585-e3ad-4a46-a462-2e7b7fc795c5 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.086130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818d585-e3ad-4a46-a462-2e7b7fc795c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.086521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818d585-e3ad-4a46-a462-2e7b7fc795c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.092347 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 429/844] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:06:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:14.097553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3300a-d57b-44a0-9422-30cb90d549bb None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:14.165999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818d585-e3ad-4a46-a462-2e7b7fc795c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:14.167492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818d585-e3ad-4a46-a462-2e7b7fc795c5 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:14.198461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:14.213805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3300a-d57b-44a0-9422-30cb90d549bb None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:14.225082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:14.253612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7bfe2-a4aa-4aa3-bc3a-d1caa3888f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:14.254249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7bfe2-a4aa-4aa3-bc3a-d1caa3888f13 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.254635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7bfe2-a4aa-4aa3-bc3a-d1caa3888f13 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.254931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7bfe2-a4aa-4aa3-bc3a-d1caa3888f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.268039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7bfe2-a4aa-4aa3-bc3a-d1caa3888f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:14.269601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7bfe2-a4aa-4aa3-bc3a-d1caa3888f13 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:14.278186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 416/845] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:06:14.285505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc82c-b3c9-4418-a490-55e7558936f9 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.285831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc82c-b3c9-4418-a490-55e7558936f9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-4ef7bfe2-a4aa-4aa3-bc3a-d1caa3888f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:14.285955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:14.287092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:14.293069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:14.293069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:14.293069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:14.293246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc82c-b3c9-4418-a490-55e7558936f9 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.293246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 417/847] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:14.293246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafc82c-b3c9-4418-a490-55e7558936f9 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:14.293448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 430/847] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:14.293448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29a3e6-c9b1-43db-adae-cac3db00d228 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.293448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29a3e6-c9b1-43db-adae-cac3db00d228 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.293939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29a3e6-c9b1-43db-adae-cac3db00d228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.314032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-ba29a3e6-c9b1-43db-adae-cac3db00d228 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:14.315239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ba29a3e6-c9b1-43db-adae-cac3db00d228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:14.387251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 431/848] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:14.388319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29a3e6-c9b1-43db-adae-cac3db00d228 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:14.435481 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafc82c-b3c9-4418-a490-55e7558936f9 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:14.477733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:14.488897 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26c38c-e57a-4d8f-babd-af2e2f6438a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:14.489543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26c38c-e57a-4d8f-babd-af2e2f6438a1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.494197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:14.496328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26c38c-e57a-4d8f-babd-af2e2f6438a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.496787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26c38c-e57a-4d8f-babd-af2e2f6438a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.503901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29a3e6-c9b1-43db-adae-cac3db00d228 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:14.509589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26c38c-e57a-4d8f-babd-af2e2f6438a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:14.511180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26c38c-e57a-4d8f-babd-af2e2f6438a1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-6c26c38c-e57a-4d8f-babd-af2e2f6438a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:14.528840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:14.530057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:14.531787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:14.531787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:14.531787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:14.532195 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 418/849] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:14.537684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26c38c-e57a-4d8f-babd-af2e2f6438a1 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:14.544534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f04b5-6ab6-4d63-ab9c-5d53f1d18144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:14.545348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f04b5-6ab6-4d63-ab9c-5d53f1d18144 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.545794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f04b5-6ab6-4d63-ab9c-5d53f1d18144 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.546821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f04b5-6ab6-4d63-ab9c-5d53f1d18144 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.547611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f04b5-6ab6-4d63-ab9c-5d53f1d18144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:14.548056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f04b5-6ab6-4d63-ab9c-5d53f1d18144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:14.549614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f04b5-6ab6-4d63-ab9c-5d53f1d18144 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:14.567031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 432/850] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:14.568220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f04b5-6ab6-4d63-ab9c-5d53f1d18144 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:14.653379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26c38c-e57a-4d8f-babd-af2e2f6438a1 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:14.676793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f04b5-6ab6-4d63-ab9c-5d53f1d18144 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:14.691747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cb2c1-5f64-4ffd-ac00-1048119f450f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:14.692277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cb2c1-5f64-4ffd-ac00-1048119f450f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.692784 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cb2c1-5f64-4ffd-ac00-1048119f450f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.692904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cb2c1-5f64-4ffd-ac00-1048119f450f None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.693446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cb2c1-5f64-4ffd-ac00-1048119f450f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:14.693744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cb2c1-5f64-4ffd-ac00-1048119f450f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:14.695165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cb2c1-5f64-4ffd-ac00-1048119f450f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:14.709155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 419/851] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:14.714056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ae587-5955-4a90-9d41-55ef1b395588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:14.714391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cb2c1-5f64-4ffd-ac00-1048119f450f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:14.714720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ae587-5955-4a90-9d41-55ef1b395588 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.714996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ae587-5955-4a90-9d41-55ef1b395588 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.715264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ae587-5955-4a90-9d41-55ef1b395588 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.724487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ae587-5955-4a90-9d41-55ef1b395588 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:14.726345 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ae587-5955-4a90-9d41-55ef1b395588 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ce4ae587-5955-4a90-9d41-55ef1b395588 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:14.738655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:14.739737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 433/852] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:14.740690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ae587-5955-4a90-9d41-55ef1b395588 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:14.758297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:14.767445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:14.827893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cb2c1-5f64-4ffd-ac00-1048119f450f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:14.852539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ae587-5955-4a90-9d41-55ef1b395588 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:14.868791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388f02c-30b2-4a49-9e84-f791cb08e92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:14.869587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388f02c-30b2-4a49-9e84-f791cb08e92a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.869867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388f02c-30b2-4a49-9e84-f791cb08e92a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.870384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388f02c-30b2-4a49-9e84-f791cb08e92a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.871379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388f02c-30b2-4a49-9e84-f791cb08e92a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:14.872753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388f02c-30b2-4a49-9e84-f791cb08e92a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-b388f02c-30b2-4a49-9e84-f791cb08e92a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:14.886980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:14.888009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:14.888838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 420/853] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:06:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:14.889630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388f02c-30b2-4a49-9e84-f791cb08e92a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:14.892736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd200e-9a39-4b13-b70f-b8fd64b4c6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:14.893389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd200e-9a39-4b13-b70f-b8fd64b4c6c3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:14.893664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd200e-9a39-4b13-b70f-b8fd64b4c6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:14.893933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd200e-9a39-4b13-b70f-b8fd64b4c6c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:14.901143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd200e-9a39-4b13-b70f-b8fd64b4c6c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:14.902626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd200e-9a39-4b13-b70f-b8fd64b4c6c3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-6dcd200e-9a39-4b13-b70f-b8fd64b4c6c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:14.914607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:14.915624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:14.916551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:14.916551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:14.916551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:14.916551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:14.916551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:14.916551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:14.916551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:14.916551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:14.916551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 434/854] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:14.916551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd200e-9a39-4b13-b70f-b8fd64b4c6c3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:14.998360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:15.000708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388f02c-30b2-4a49-9e84-f791cb08e92a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:15.004016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:15.034960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd200e-9a39-4b13-b70f-b8fd64b4c6c3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:15.037262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd930e5c-5c06-4aa9-9359-779ae51cc8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:15.037873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd930e5c-5c06-4aa9-9359-779ae51cc8b3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:15.038152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd930e5c-5c06-4aa9-9359-779ae51cc8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:15.038429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd930e5c-5c06-4aa9-9359-779ae51cc8b3 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:15.039196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd930e5c-5c06-4aa9-9359-779ae51cc8b3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:15.039503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd930e5c-5c06-4aa9-9359-779ae51cc8b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:15.040947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd930e5c-5c06-4aa9-9359-779ae51cc8b3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:15.059427 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 421/855] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 22 17:06:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:15.060314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd930e5c-5c06-4aa9-9359-779ae51cc8b3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:15.080843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba9b14-1766-45e3-b36c-6dc4165ebee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:15.081456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9b14-1766-45e3-b36c-6dc4165ebee6 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:15.081733 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9b14-1766-45e3-b36c-6dc4165ebee6 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:15.081999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9b14-1766-45e3-b36c-6dc4165ebee6 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:15.082706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba9b14-1766-45e3-b36c-6dc4165ebee6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:15.082955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba9b14-1766-45e3-b36c-6dc4165ebee6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:15.084350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba9b14-1766-45e3-b36c-6dc4165ebee6 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:15.094532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 435/856] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:06:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:15.095636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba9b14-1766-45e3-b36c-6dc4165ebee6 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:15.164341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd930e5c-5c06-4aa9-9359-779ae51cc8b3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:15.197718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba9b14-1766-45e3-b36c-6dc4165ebee6 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:15.199104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c46aa-0ce4-4675-9e2f-0130cdcc9fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:15.199706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c46aa-0ce4-4675-9e2f-0130cdcc9fb0 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:15.200003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c46aa-0ce4-4675-9e2f-0130cdcc9fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:15.200277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c46aa-0ce4-4675-9e2f-0130cdcc9fb0 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:15.200960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c46aa-0ce4-4675-9e2f-0130cdcc9fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:15.201263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c46aa-0ce4-4675-9e2f-0130cdcc9fb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:15.202873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c46aa-0ce4-4675-9e2f-0130cdcc9fb0 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:15.214802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 422/857] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:15.215618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c46aa-0ce4-4675-9e2f-0130cdcc9fb0 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:15.236556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92da7bf-6c37-4775-a3a4-77c79ad1e6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:15.237169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92da7bf-6c37-4775-a3a4-77c79ad1e6de None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:15.237442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92da7bf-6c37-4775-a3a4-77c79ad1e6de None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:15.237713 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92da7bf-6c37-4775-a3a4-77c79ad1e6de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:15.240487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:15.241205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:15.307511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92da7bf-6c37-4775-a3a4-77c79ad1e6de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:15.308942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92da7bf-6c37-4775-a3a4-77c79ad1e6de None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:15.316258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c46aa-0ce4-4675-9e2f-0130cdcc9fb0 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:15.351641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed93f7-c6ee-49c1-96ac-e53dc8606074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:15.352250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed93f7-c6ee-49c1-96ac-e53dc8606074 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:15.352528 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed93f7-c6ee-49c1-96ac-e53dc8606074 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:15.352797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed93f7-c6ee-49c1-96ac-e53dc8606074 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:15.353431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed93f7-c6ee-49c1-96ac-e53dc8606074 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:15.353733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed93f7-c6ee-49c1-96ac-e53dc8606074 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:15.355147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed93f7-c6ee-49c1-96ac-e53dc8606074 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:15.359685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 423/858] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:06:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:15.360853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf449ed-fe6e-483f-84f6-e992d6b34420 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:15.361134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf449ed-fe6e-483f-84f6-e992d6b34420 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:15.361404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf449ed-fe6e-483f-84f6-e992d6b34420 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:15.362658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 424/859] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:15.363412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf449ed-fe6e-483f-84f6-e992d6b34420 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:15.441450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 436/860] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:06:15.442690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56b428-5740-4a7d-b2e7-92bdf2e0e62a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:15.442979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56b428-5740-4a7d-b2e7-92bdf2e0e62a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:15.443405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56b428-5740-4a7d-b2e7-92bdf2e0e62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:15.465066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-ea56b428-5740-4a7d-b2e7-92bdf2e0e62a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:15.466294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ea56b428-5740-4a7d-b2e7-92bdf2e0e62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:15.468870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:15.486962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf449ed-fe6e-483f-84f6-e992d6b34420 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:15.502018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:15.532908 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e2bd-3297-40c5-b2b4-536867131fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:15.533869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e2bd-3297-40c5-b2b4-536867131fe3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:15.534166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e2bd-3297-40c5-b2b4-536867131fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:15.534442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e2bd-3297-40c5-b2b4-536867131fe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:15.535281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309e2bd-3297-40c5-b2b4-536867131fe3 None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=9001ee719e294f79b35f13389ac9cbaa) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:15.535592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309e2bd-3297-40c5-b2b4-536867131fe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:15.537155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309e2bd-3297-40c5-b2b4-536867131fe3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:15.543629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 425/861] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:15 2022] GET /identity/v3/role_assignments?user.id=9001ee719e294f79b35f13389ac9cbaa&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:15.550817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e2bd-3297-40c5-b2b4-536867131fe3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:15.554446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 437/862] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:15.682557 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e2bd-3297-40c5-b2b4-536867131fe3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:15.723805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e47d8-b656-4a81-af4e-151ba67ee51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:15.724462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e47d8-b656-4a81-af4e-151ba67ee51e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:15.724726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e47d8-b656-4a81-af4e-151ba67ee51e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:15.724997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e47d8-b656-4a81-af4e-151ba67ee51e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:15.736352 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e47d8-b656-4a81-af4e-151ba67ee51e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:15.737824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e47d8-b656-4a81-af4e-151ba67ee51e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-873e47d8-b656-4a81-af4e-151ba67ee51e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:15.753674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:15.754735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:15.755770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:15.755770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:15.755770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:15.755770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 426/863] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:15.762034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56b428-5740-4a7d-b2e7-92bdf2e0e62a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:15.833916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e47d8-b656-4a81-af4e-151ba67ee51e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:15.837194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:15.883658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56b428-5740-4a7d-b2e7-92bdf2e0e62a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:15.919832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767b6ad-0d6c-43c1-9509-4b1437567627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:15.920444 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767b6ad-0d6c-43c1-9509-4b1437567627 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:15.920722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767b6ad-0d6c-43c1-9509-4b1437567627 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:15.921095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767b6ad-0d6c-43c1-9509-4b1437567627 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:15.921969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767b6ad-0d6c-43c1-9509-4b1437567627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:15.922792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767b6ad-0d6c-43c1-9509-4b1437567627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:15.924213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767b6ad-0d6c-43c1-9509-4b1437567627 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:15.937360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 438/864] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:15.943288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767b6ad-0d6c-43c1-9509-4b1437567627 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:15.976381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:15.989764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e47d8-b656-4a81-af4e-151ba67ee51e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:16.031594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6487c86-e2fb-434c-9498-bf44c6ad1924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:16.032308 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6487c86-e2fb-434c-9498-bf44c6ad1924 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:16.032575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6487c86-e2fb-434c-9498-bf44c6ad1924 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:16.032920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6487c86-e2fb-434c-9498-bf44c6ad1924 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:16.033431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6487c86-e2fb-434c-9498-bf44c6ad1924 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:16.034846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6487c86-e2fb-434c-9498-bf44c6ad1924 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-c6487c86-e2fb-434c-9498-bf44c6ad1924 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:16.044805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:16.045820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 17:06:16.047390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:16.051831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 427/865] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Aug 22 17:06:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:16.052596 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6487c86-e2fb-434c-9498-bf44c6ad1924 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:16.085596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767b6ad-0d6c-43c1-9509-4b1437567627 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:16.130080 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fcbfc-8e93-45fc-8c8f-b8db0c6feead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:16.130943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fcbfc-8e93-45fc-8c8f-b8db0c6feead None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:16.131399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fcbfc-8e93-45fc-8c8f-b8db0c6feead None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:16.131740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fcbfc-8e93-45fc-8c8f-b8db0c6feead None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:16.136431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:16.141971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fcbfc-8e93-45fc-8c8f-b8db0c6feead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:16.144662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fcbfc-8e93-45fc-8c8f-b8db0c6feead None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-a01fcbfc-8e93-45fc-8c8f-b8db0c6feead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:16.158930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:16.159905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:16.160868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:16.160868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:16.160868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:16.160868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:16.160868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:16.160868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:16.160868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:16.160868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:16.162554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 439/866] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:16.165810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fcbfc-8e93-45fc-8c8f-b8db0c6feead None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:16.194088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6487c86-e2fb-434c-9498-bf44c6ad1924 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:16.242469 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19e46f-1953-4301-b128-92309228a651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:16.243397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19e46f-1953-4301-b128-92309228a651 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:16.244081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19e46f-1953-4301-b128-92309228a651 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:16.244355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19e46f-1953-4301-b128-92309228a651 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:16.245273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19e46f-1953-4301-b128-92309228a651 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:16.245576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19e46f-1953-4301-b128-92309228a651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:16.246979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19e46f-1953-4301-b128-92309228a651 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:16.255189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 428/867] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:16.263809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19e46f-1953-4301-b128-92309228a651 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:16.275386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:16.298426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fcbfc-8e93-45fc-8c8f-b8db0c6feead None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:16.344219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb786fc-332f-4fa2-b568-7a880a0eea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:16.345319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb786fc-332f-4fa2-b568-7a880a0eea65 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:16.345777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb786fc-332f-4fa2-b568-7a880a0eea65 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:16.346348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb786fc-332f-4fa2-b568-7a880a0eea65 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:16.347928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb786fc-332f-4fa2-b568-7a880a0eea65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:16.348314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb786fc-332f-4fa2-b568-7a880a0eea65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:16.350229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb786fc-332f-4fa2-b568-7a880a0eea65 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:16.370659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 440/868] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:16.373839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb786fc-332f-4fa2-b568-7a880a0eea65 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:16.380681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19e46f-1953-4301-b128-92309228a651 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:16.413206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:16.417078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159754e-8fa7-4c4d-8080-1ac3b940a87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:16.417815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159754e-8fa7-4c4d-8080-1ac3b940a87c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:16.418186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159754e-8fa7-4c4d-8080-1ac3b940a87c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:16.418599 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159754e-8fa7-4c4d-8080-1ac3b940a87c None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:16.419647 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159754e-8fa7-4c4d-8080-1ac3b940a87c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:16.420015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159754e-8fa7-4c4d-8080-1ac3b940a87c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:16.422507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159754e-8fa7-4c4d-8080-1ac3b940a87c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:16.434508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 429/869] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:16.508321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb786fc-332f-4fa2-b568-7a880a0eea65 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:16.550041 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62223435-7c75-4350-bd06-60a4d1a3cf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:16.550874 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62223435-7c75-4350-bd06-60a4d1a3cf76 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:16.551245 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62223435-7c75-4350-bd06-60a4d1a3cf76 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:16.551612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62223435-7c75-4350-bd06-60a4d1a3cf76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:16.552453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62223435-7c75-4350-bd06-60a4d1a3cf76 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=38462b1165514ed7b948cba710782d4b) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:16.553134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62223435-7c75-4350-bd06-60a4d1a3cf76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:16.554714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62223435-7c75-4350-bd06-60a4d1a3cf76 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:16.562631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 441/870] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:16 2022] GET /identity/v3/role_assignments?user.id=38462b1165514ed7b948cba710782d4b&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=4b10782d99b74f5bb453b82286012e0c => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:17.340648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee0820-4b68-4b4e-9619-4313c479c863 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:17.341180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee0820-4b68-4b4e-9619-4313c479c863 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:17.341180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee0820-4b68-4b4e-9619-4313c479c863 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:17.343570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 430/871] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:17.349627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfb5c8-c160-4b63-ab41-b64512b4dad2 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:17.349913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfb5c8-c160-4b63-ab41-b64512b4dad2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:17.350183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfb5c8-c160-4b63-ab41-b64512b4dad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:17.369923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-dbcfb5c8-c160-4b63-ab41-b64512b4dad2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:17.371267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dbcfb5c8-c160-4b63-ab41-b64512b4dad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:17.373309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:17.446606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 442/872] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:17.690674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580a2bf-e872-47ab-a34f-999efff45d7e None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:17.690972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580a2bf-e872-47ab-a34f-999efff45d7e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:17.691219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580a2bf-e872-47ab-a34f-999efff45d7e None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:17.692863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 431/873] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:17.698327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08802a-4383-4678-90a3-eaff35728f4b None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:17.698811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08802a-4383-4678-90a3-eaff35728f4b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:17.698990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08802a-4383-4678-90a3-eaff35728f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:17.718572 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1b08802a-4383-4678-90a3-eaff35728f4b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:17.720034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1b08802a-4383-4678-90a3-eaff35728f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:17.723290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580a2bf-e872-47ab-a34f-999efff45d7e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:17.826506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 443/874] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:17.839401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580a2bf-e872-47ab-a34f-999efff45d7e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:17.876817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e280b-f897-4578-8264-d8e13490baa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:17.877343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e280b-f897-4578-8264-d8e13490baa4 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:17.877620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e280b-f897-4578-8264-d8e13490baa4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:17.877890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e280b-f897-4578-8264-d8e13490baa4 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:17.878552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e280b-f897-4578-8264-d8e13490baa4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:17.880035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e280b-f897-4578-8264-d8e13490baa4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:17.953082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 432/875] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:06:17 2022] POST /identity/v3/services => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:18.108690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08802a-4383-4678-90a3-eaff35728f4b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:18.184169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:18.220274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08802a-4383-4678-90a3-eaff35728f4b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:18.271759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57a13a-10a1-44e0-904d-eedaa31a1cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:18.272634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57a13a-10a1-44e0-904d-eedaa31a1cde None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:18.272916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57a13a-10a1-44e0-904d-eedaa31a1cde None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:18.273219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57a13a-10a1-44e0-904d-eedaa31a1cde None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:18.285532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57a13a-10a1-44e0-904d-eedaa31a1cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-7d57a13a-10a1-44e0-904d-eedaa31a1cde None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:18.290217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:18.291215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:18.292057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:18.297268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 444/876] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 22 17:06:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:18.299597 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e280b-f897-4578-8264-d8e13490baa4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:18.363006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:18.412746 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e280b-f897-4578-8264-d8e13490baa4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:18.423341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8c0dd-8b82-4cc5-9af5-f28e64a2c999 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:18.423619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8c0dd-8b82-4cc5-9af5-f28e64a2c999 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:18.423874 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8c0dd-8b82-4cc5-9af5-f28e64a2c999 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:18.426056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 445/877] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:18.431249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c57f58-2e81-476d-9a60-84ef1341cb55 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:18.431531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c57f58-2e81-476d-9a60-84ef1341cb55 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:18.431799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c57f58-2e81-476d-9a60-84ef1341cb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:18.453019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c5c57f58-2e81-476d-9a60-84ef1341cb55 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:18.453650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417cf1b-4874-47c4-b55d-47b897717980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:18.454246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c5c57f58-2e81-476d-9a60-84ef1341cb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:18.454862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417cf1b-4874-47c4-b55d-47b897717980 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:18.455332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417cf1b-4874-47c4-b55d-47b897717980 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:18.455773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417cf1b-4874-47c4-b55d-47b897717980 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:18.456798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417cf1b-4874-47c4-b55d-47b897717980 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:18.458116 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417cf1b-4874-47c4-b55d-47b897717980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:18.459768 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417cf1b-4874-47c4-b55d-47b897717980 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:18.470854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 433/878] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 22 17:06:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:18.485241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417cf1b-4874-47c4-b55d-47b897717980 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:18.542611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 446/879] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:18 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:18.543887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c3b0f-0a6a-49fb-8490-116d3c25f505 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:18.544421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c3b0f-0a6a-49fb-8490-116d3c25f505 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:18.544691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c3b0f-0a6a-49fb-8490-116d3c25f505 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:18.546634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 447/880] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:18.552540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98776f2f-edd6-48ab-a8fd-5e59ed7617a3 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:18.552823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98776f2f-edd6-48ab-a8fd-5e59ed7617a3 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:18.553098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98776f2f-edd6-48ab-a8fd-5e59ed7617a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:18.577115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-98776f2f-edd6-48ab-a8fd-5e59ed7617a3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:18.578457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-98776f2f-edd6-48ab-a8fd-5e59ed7617a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:18.612175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417cf1b-4874-47c4-b55d-47b897717980 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:18.635976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:18.647377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:18.656366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd6f35-a6aa-4621-a860-9a4c4f8878c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:18.657297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd6f35-a6aa-4621-a860-9a4c4f8878c3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:18.657755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd6f35-a6aa-4621-a860-9a4c4f8878c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:18.658206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd6f35-a6aa-4621-a860-9a4c4f8878c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:18.670617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 448/881] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:18 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:18.676342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd6f35-a6aa-4621-a860-9a4c4f8878c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:18.678064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd6f35-a6aa-4621-a860-9a4c4f8878c3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-03cd6f35-a6aa-4621-a860-9a4c4f8878c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:18.694079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:18.695146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:18.696055 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:18.696055 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:18.696055 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:18.697892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 434/882] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:18.707841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98776f2f-edd6-48ab-a8fd-5e59ed7617a3 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:18.817624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98776f2f-edd6-48ab-a8fd-5e59ed7617a3 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:18.840879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd6f35-a6aa-4621-a860-9a4c4f8878c3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:18.854427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde207f8-f3f5-4f9a-9f86-9a6c8635bb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:18.855064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde207f8-f3f5-4f9a-9f86-9a6c8635bb93 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:18.855339 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde207f8-f3f5-4f9a-9f86-9a6c8635bb93 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:18.855612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde207f8-f3f5-4f9a-9f86-9a6c8635bb93 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:18.856255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde207f8-f3f5-4f9a-9f86-9a6c8635bb93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:18.856554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde207f8-f3f5-4f9a-9f86-9a6c8635bb93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:18.858117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde207f8-f3f5-4f9a-9f86-9a6c8635bb93 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:18.871856 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 449/883] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:18.879744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde207f8-f3f5-4f9a-9f86-9a6c8635bb93 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:18.941575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:18.971016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd6f35-a6aa-4621-a860-9a4c4f8878c3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:19.010569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35f993-2dfa-479b-8a78-1e3f3872c39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:19.010816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde207f8-f3f5-4f9a-9f86-9a6c8635bb93 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:19.011184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35f993-2dfa-479b-8a78-1e3f3872c39d None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:19.011462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35f993-2dfa-479b-8a78-1e3f3872c39d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:19.011803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35f993-2dfa-479b-8a78-1e3f3872c39d None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:19.012393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35f993-2dfa-479b-8a78-1e3f3872c39d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:19.013808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35f993-2dfa-479b-8a78-1e3f3872c39d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:19.046772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59454dbd-0472-4a51-95eb-8d199fd284bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:19.047413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59454dbd-0472-4a51-95eb-8d199fd284bb None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:19.047691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59454dbd-0472-4a51-95eb-8d199fd284bb None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:19.048043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59454dbd-0472-4a51-95eb-8d199fd284bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:19.055982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59454dbd-0472-4a51-95eb-8d199fd284bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:19.057429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59454dbd-0472-4a51-95eb-8d199fd284bb None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-59454dbd-0472-4a51-95eb-8d199fd284bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:19.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:19.069985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:19.070994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:19.070994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:19.070994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:19.070994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:19.070994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:19.070994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:19.070994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:19.070994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:19.070994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:19.070994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:19.074641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 450/884] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 22 17:06:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:19.075682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59454dbd-0472-4a51-95eb-8d199fd284bb None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:19.118959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 435/885] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:06:18 2022] POST /identity/v3/services => generated 269 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:19.120036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35f993-2dfa-479b-8a78-1e3f3872c39d None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:19.159838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:19.211247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59454dbd-0472-4a51-95eb-8d199fd284bb None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:19.214841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:19.269330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e185ce-8e55-4ab2-a337-df218e1838fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:19.269904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e185ce-8e55-4ab2-a337-df218e1838fb None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:19.270288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e185ce-8e55-4ab2-a337-df218e1838fb None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:19.270582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e185ce-8e55-4ab2-a337-df218e1838fb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:19.271075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35f993-2dfa-479b-8a78-1e3f3872c39d None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:19.271252 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e185ce-8e55-4ab2-a337-df218e1838fb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:19.272520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e185ce-8e55-4ab2-a337-df218e1838fb None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-a8e185ce-8e55-4ab2-a337-df218e1838fb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:19.284735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:19.285784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:19.286709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:19.288836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 451/886] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:19.295064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e185ce-8e55-4ab2-a337-df218e1838fb None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:19.314774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf43d5-9057-425a-893b-9b441dc5a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:19.316136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf43d5-9057-425a-893b-9b441dc5a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:19.316136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf43d5-9057-425a-893b-9b441dc5a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:19.316136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf43d5-9057-425a-893b-9b441dc5a8f6 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:19.316631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf43d5-9057-425a-893b-9b441dc5a8f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:19.316943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf43d5-9057-425a-893b-9b441dc5a8f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:19.319182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf43d5-9057-425a-893b-9b441dc5a8f6 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:19.338199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 436/887] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 22 17:06:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:19.345136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf43d5-9057-425a-893b-9b441dc5a8f6 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:19.421546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e185ce-8e55-4ab2-a337-df218e1838fb None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:19.461462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333158c3-a8f7-488d-b9d3-88bbc0464f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:19.462491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333158c3-a8f7-488d-b9d3-88bbc0464f39 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:19.462993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333158c3-a8f7-488d-b9d3-88bbc0464f39 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:19.463463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333158c3-a8f7-488d-b9d3-88bbc0464f39 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:19.464276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333158c3-a8f7-488d-b9d3-88bbc0464f39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:19.464381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:19.465244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333158c3-a8f7-488d-b9d3-88bbc0464f39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:19.466851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333158c3-a8f7-488d-b9d3-88bbc0464f39 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:19.473520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 452/888] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 22 17:06:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:19.479454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333158c3-a8f7-488d-b9d3-88bbc0464f39 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:19.499811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf43d5-9057-425a-893b-9b441dc5a8f6 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:19.518132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:19.542297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0403d13-0cef-49f5-a54f-0df724e8bb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:19.543253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0403d13-0cef-49f5-a54f-0df724e8bb31 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:19.543543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0403d13-0cef-49f5-a54f-0df724e8bb31 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:19.543819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0403d13-0cef-49f5-a54f-0df724e8bb31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:19.609851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333158c3-a8f7-488d-b9d3-88bbc0464f39 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:19.633597 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0403d13-0cef-49f5-a54f-0df724e8bb31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:19.635153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0403d13-0cef-49f5-a54f-0df724e8bb31 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:19.652969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870bf3a-b919-4c5b-a3b9-a32f52d2821d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:19.653919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870bf3a-b919-4c5b-a3b9-a32f52d2821d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:19.654407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870bf3a-b919-4c5b-a3b9-a32f52d2821d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:19.654918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870bf3a-b919-4c5b-a3b9-a32f52d2821d None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:19.656802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870bf3a-b919-4c5b-a3b9-a32f52d2821d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:19.657322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870bf3a-b919-4c5b-a3b9-a32f52d2821d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:19.658955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870bf3a-b919-4c5b-a3b9-a32f52d2821d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:19.664674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 453/889] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 22 17:06:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:19.732937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:19.756186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 437/890] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:19 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 17:06:19.761889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870bf3a-b919-4c5b-a3b9-a32f52d2821d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:19.867803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:19.876827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870bf3a-b919-4c5b-a3b9-a32f52d2821d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:19.912086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a66fff-731a-44a5-8fa5-5b6628b8e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:19.912727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66fff-731a-44a5-8fa5-5b6628b8e5fd None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:19.913007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66fff-731a-44a5-8fa5-5b6628b8e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:19.913279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66fff-731a-44a5-8fa5-5b6628b8e5fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:19.925767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a66fff-731a-44a5-8fa5-5b6628b8e5fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:19.927327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a66fff-731a-44a5-8fa5-5b6628b8e5fd None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-26a66fff-731a-44a5-8fa5-5b6628b8e5fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:19.942727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:19.943723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:19.945199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:19.945199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:19.945199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:19.945199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 454/891] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:19.951891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0403d13-0cef-49f5-a54f-0df724e8bb31 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:19.982001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510b07-d23c-43de-94e6-87c69121dfbe None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:19.982332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510b07-d23c-43de-94e6-87c69121dfbe None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:19.982580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510b07-d23c-43de-94e6-87c69121dfbe None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:19.984704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 455/892] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:19.989891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fd1c9-b2b1-429e-a069-565a19f4e109 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:19.990174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fd1c9-b2b1-429e-a069-565a19f4e109 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:19.990446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fd1c9-b2b1-429e-a069-565a19f4e109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:20.010135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-400fd1c9-b2b1-429e-a069-565a19f4e109 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:20.011373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-400fd1c9-b2b1-429e-a069-565a19f4e109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:20.069909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0403d13-0cef-49f5-a54f-0df724e8bb31 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:20.093759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 456/893] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:20.113549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ed439-e8fb-4f7b-84c8-a2f228caeb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:20.114624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed439-e8fb-4f7b-84c8-a2f228caeb1b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:20.115103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed439-e8fb-4f7b-84c8-a2f228caeb1b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:20.115599 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed439-e8fb-4f7b-84c8-a2f228caeb1b None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:20.116462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ed439-e8fb-4f7b-84c8-a2f228caeb1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:20.116992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ed439-e8fb-4f7b-84c8-a2f228caeb1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:20.118720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ed439-e8fb-4f7b-84c8-a2f228caeb1b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:20.131986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 438/894] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:20.137173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fd1c9-b2b1-429e-a069-565a19f4e109 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:20.219697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:20.246032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fd1c9-b2b1-429e-a069-565a19f4e109 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:20.283415 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daa67c-2cf7-4fab-92fd-e88e3e3a786a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:20.284433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daa67c-2cf7-4fab-92fd-e88e3e3a786a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:20.284912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daa67c-2cf7-4fab-92fd-e88e3e3a786a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:20.285403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daa67c-2cf7-4fab-92fd-e88e3e3a786a None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:20.292652 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89daa67c-2cf7-4fab-92fd-e88e3e3a786a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:20.294662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89daa67c-2cf7-4fab-92fd-e88e3e3a786a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-89daa67c-2cf7-4fab-92fd-e88e3e3a786a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:20.306573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:20.307550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:20.308471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:20.308471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:20.308471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:20.308471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:20.308471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:20.308471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:20.308471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:20.308471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:20.311139 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 457/895] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:20.320048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ed439-e8fb-4f7b-84c8-a2f228caeb1b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:20.370559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:20.376083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daa67c-2cf7-4fab-92fd-e88e3e3a786a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:20.444642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ed439-e8fb-4f7b-84c8-a2f228caeb1b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:20.486216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daa67c-2cf7-4fab-92fd-e88e3e3a786a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:20.491659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba6b5-cd33-4074-8f3d-90462c30ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:20.492939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba6b5-cd33-4074-8f3d-90462c30ef99 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:20.493477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba6b5-cd33-4074-8f3d-90462c30ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:20.493962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba6b5-cd33-4074-8f3d-90462c30ef99 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:20.495307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ba6b5-cd33-4074-8f3d-90462c30ef99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:20.495630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ba6b5-cd33-4074-8f3d-90462c30ef99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:20.497089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ba6b5-cd33-4074-8f3d-90462c30ef99 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:20.516063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 439/896] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:20.516976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba6b5-cd33-4074-8f3d-90462c30ef99 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:20.532420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660a62c-fe99-4cfa-b4be-859282521293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:20.533583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660a62c-fe99-4cfa-b4be-859282521293 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:20.534065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660a62c-fe99-4cfa-b4be-859282521293 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:20.534999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660a62c-fe99-4cfa-b4be-859282521293 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:20.536040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660a62c-fe99-4cfa-b4be-859282521293 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:20.537774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660a62c-fe99-4cfa-b4be-859282521293 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:20.540167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-b660a62c-fe99-4cfa-b4be-859282521293 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:20.549955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:20.553668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 17:06:20.554522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:20.556438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 458/897] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:06:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:20.557420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660a62c-fe99-4cfa-b4be-859282521293 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:20.647618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba6b5-cd33-4074-8f3d-90462c30ef99 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:20.649461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:20.677489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660a62c-fe99-4cfa-b4be-859282521293 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:20.686025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bb9e6-ce0e-4186-85b1-dd69f0771f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:20.686900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bb9e6-ce0e-4186-85b1-dd69f0771f7f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:20.687252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bb9e6-ce0e-4186-85b1-dd69f0771f7f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:20.687593 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bb9e6-ce0e-4186-85b1-dd69f0771f7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:20.688579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bb9e6-ce0e-4186-85b1-dd69f0771f7f None admin] RBAC: Adding query filter params (role.id=ccb62dfdcc3947e9bdb4bba0f88e4e10, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=9001ee719e294f79b35f13389ac9cbaa) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:20.688812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bb9e6-ce0e-4186-85b1-dd69f0771f7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:20.690399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bb9e6-ce0e-4186-85b1-dd69f0771f7f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:20.697534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 440/898] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:20 2022] GET /identity/v3/role_assignments?user.id=9001ee719e294f79b35f13389ac9cbaa&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=ccb62dfdcc3947e9bdb4bba0f88e4e10 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:20.717948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d978d-e89a-400e-a475-215acafbd000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:20.718880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d978d-e89a-400e-a475-215acafbd000 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:20.719304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d978d-e89a-400e-a475-215acafbd000 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:20.719724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d978d-e89a-400e-a475-215acafbd000 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:20.720533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d978d-e89a-400e-a475-215acafbd000 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:20.721034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d978d-e89a-400e-a475-215acafbd000 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:20.722645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d978d-e89a-400e-a475-215acafbd000 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:20.728919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 459/899] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:06:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:20.735137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bb9e6-ce0e-4186-85b1-dd69f0771f7f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:20.853998 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bb9e6-ce0e-4186-85b1-dd69f0771f7f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:20.895763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1034c-97a0-4044-a5d9-c4b9dd8e6fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:20.896667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1034c-97a0-4044-a5d9-c4b9dd8e6fac None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:20.897102 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1034c-97a0-4044-a5d9-c4b9dd8e6fac None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:20.897395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1034c-97a0-4044-a5d9-c4b9dd8e6fac None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:20.898036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1034c-97a0-4044-a5d9-c4b9dd8e6fac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:20.898334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1034c-97a0-4044-a5d9-c4b9dd8e6fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:20.899765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1034c-97a0-4044-a5d9-c4b9dd8e6fac None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:20.905366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 441/900] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:06:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:20.912969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d978d-e89a-400e-a475-215acafbd000 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:20.980477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:21.021622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d978d-e89a-400e-a475-215acafbd000 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:21.058316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d3165-b647-4e57-9142-4ffaeabcc19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:21.058962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d3165-b647-4e57-9142-4ffaeabcc19f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:21.059238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d3165-b647-4e57-9142-4ffaeabcc19f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:21.059505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d3165-b647-4e57-9142-4ffaeabcc19f None admin] PATH_INFO: `/v3/endpoints` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:21.060208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d3165-b647-4e57-9142-4ffaeabcc19f None admin] RBAC: Adding query filter params (interface=public, service_id=8f0ddd5191904d808b4cd137bcdfea44, region_id=RegionOne) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:21.060580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d3165-b647-4e57-9142-4ffaeabcc19f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:21.062544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d3165-b647-4e57-9142-4ffaeabcc19f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:21.068707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 460/901] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 22 17:06:20 2022] GET /identity/v3/endpoints?service_id=8f0ddd5191904d808b4cd137bcdfea44&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:21.074746 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1034c-97a0-4044-a5d9-c4b9dd8e6fac None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:21.110758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:21.185839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1034c-97a0-4044-a5d9-c4b9dd8e6fac None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:21.207173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab110d3-6ac7-423b-8656-d0e6a7e269bd None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:21.207477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab110d3-6ac7-423b-8656-d0e6a7e269bd None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:21.207719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab110d3-6ac7-423b-8656-d0e6a7e269bd None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:21.209354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 461/902] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:21.214600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a7b4e-21fa-45bf-90ab-6effb07523b3 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:21.214884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a7b4e-21fa-45bf-90ab-6effb07523b3 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:21.215155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a7b4e-21fa-45bf-90ab-6effb07523b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:21.224675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a10bd-b87f-44d0-852a-db2f1055e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:21.225289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a10bd-b87f-44d0-852a-db2f1055e821 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:21.225567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a10bd-b87f-44d0-852a-db2f1055e821 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:21.225843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a10bd-b87f-44d0-852a-db2f1055e821 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:21.226458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a10bd-b87f-44d0-852a-db2f1055e821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:21.227863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a10bd-b87f-44d0-852a-db2f1055e821 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:21.235270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d10a7b4e-21fa-45bf-90ab-6effb07523b3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:21.235663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 442/903] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 22 17:06:21 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:06:21.236536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d10a7b4e-21fa-45bf-90ab-6effb07523b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:21.339665 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 462/904] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:21.643144 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a10bd-b87f-44d0-852a-db2f1055e821 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:21.665067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6171dd-a7d2-407d-b690-1f116a0ae48f None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:21.665351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6171dd-a7d2-407d-b690-1f116a0ae48f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:21.665613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6171dd-a7d2-407d-b690-1f116a0ae48f None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:21.667271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 463/905] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:21.673241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e6da-2f75-4562-8ab7-084691492421 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:21.673520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e6da-2f75-4562-8ab7-084691492421 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:21.673795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e6da-2f75-4562-8ab7-084691492421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:21.692578 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-df07e6da-2f75-4562-8ab7-084691492421 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:21.693763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-df07e6da-2f75-4562-8ab7-084691492421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:21.746753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:21.756273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a10bd-b87f-44d0-852a-db2f1055e821 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:21.761357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:21.782273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 464/906] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:21.798142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac818e35-82d8-4516-b5f3-9e5b7042828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:21.799081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818e35-82d8-4516-b5f3-9e5b7042828c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:21.799536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818e35-82d8-4516-b5f3-9e5b7042828c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:21.800008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818e35-82d8-4516-b5f3-9e5b7042828c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:21.800769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac818e35-82d8-4516-b5f3-9e5b7042828c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:21.802354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac818e35-82d8-4516-b5f3-9e5b7042828c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-ac818e35-82d8-4516-b5f3-9e5b7042828c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:21.812500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:21.813508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 17:06:21.814498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:21.817379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 443/907] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Aug 22 17:06:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:21.823817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07e6da-2f75-4562-8ab7-084691492421 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:21.930701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07e6da-2f75-4562-8ab7-084691492421 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:21.970672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e10861-740a-4035-b281-b4493c17128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:21.971286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e10861-740a-4035-b281-b4493c17128d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:21.971560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e10861-740a-4035-b281-b4493c17128d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:21.971828 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e10861-740a-4035-b281-b4493c17128d None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:21.972454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e10861-740a-4035-b281-b4493c17128d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:21.972747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e10861-740a-4035-b281-b4493c17128d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:21.974102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e10861-740a-4035-b281-b4493c17128d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:21.984260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 465/908] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:21.985678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac818e35-82d8-4516-b5f3-9e5b7042828c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:22.068960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e10861-740a-4035-b281-b4493c17128d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:22.100587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac818e35-82d8-4516-b5f3-9e5b7042828c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:22.130877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:22.140957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc161136-11ca-450c-a533-26d745059c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:22.141612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc161136-11ca-450c-a533-26d745059c46 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:22.141891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc161136-11ca-450c-a533-26d745059c46 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:22.142158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc161136-11ca-450c-a533-26d745059c46 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:22.142892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc161136-11ca-450c-a533-26d745059c46 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:22.143192 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc161136-11ca-450c-a533-26d745059c46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:22.144691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc161136-11ca-450c-a533-26d745059c46 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:22.149988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 444/909] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:22.156769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc161136-11ca-450c-a533-26d745059c46 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:22.177767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e10861-740a-4035-b281-b4493c17128d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:22.184776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:22.215474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47310fcb-2a54-4529-821a-cc7c489673f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:22.216087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47310fcb-2a54-4529-821a-cc7c489673f2 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:22.216366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47310fcb-2a54-4529-821a-cc7c489673f2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:22.216635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47310fcb-2a54-4529-821a-cc7c489673f2 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:22.217294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47310fcb-2a54-4529-821a-cc7c489673f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:22.219468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47310fcb-2a54-4529-821a-cc7c489673f2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:22.271315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc161136-11ca-450c-a533-26d745059c46 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:22.299066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 466/910] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:06:22 2022] POST /identity/v3/services => generated 241 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:06:22.324023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab2fe0-6fed-47af-96d4-4da4c5afad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:22.325047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab2fe0-6fed-47af-96d4-4da4c5afad04 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:22.325496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab2fe0-6fed-47af-96d4-4da4c5afad04 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:22.325914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab2fe0-6fed-47af-96d4-4da4c5afad04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:22.327501 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab2fe0-6fed-47af-96d4-4da4c5afad04 None admin] RBAC: Adding query filter params (interface=public, service_id=e348af23fa39489ea24804acba7f97f2, region_id=RegionOne) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:22.328263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab2fe0-6fed-47af-96d4-4da4c5afad04 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:22.330102 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab2fe0-6fed-47af-96d4-4da4c5afad04 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:22.341161 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 445/911] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 22 17:06:22 2022] GET /identity/v3/endpoints?service_id=e348af23fa39489ea24804acba7f97f2&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:22.344709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47310fcb-2a54-4529-821a-cc7c489673f2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:22.462246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47310fcb-2a54-4529-821a-cc7c489673f2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:22.473805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:22.504449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff48f96-3b72-440d-8e12-46677f981b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:22.505065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff48f96-3b72-440d-8e12-46677f981b6b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:22.505358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff48f96-3b72-440d-8e12-46677f981b6b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:22.505610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff48f96-3b72-440d-8e12-46677f981b6b None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:22.506216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff48f96-3b72-440d-8e12-46677f981b6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:22.507682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff48f96-3b72-440d-8e12-46677f981b6b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:22.517123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 467/912] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 22 17:06:22 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:06:22.812482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e414813-c319-430b-b657-72fd9e2babe3 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:22.812978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e414813-c319-430b-b657-72fd9e2babe3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:22.812978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e414813-c319-430b-b657-72fd9e2babe3 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:22.814571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 446/913] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:22.819981 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599bc04-4812-41ed-bf79-7f37c50ac31c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:22.820266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599bc04-4812-41ed-bf79-7f37c50ac31c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:22.820545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599bc04-4812-41ed-bf79-7f37c50ac31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:22.840952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-5599bc04-4812-41ed-bf79-7f37c50ac31c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:22.842151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5599bc04-4812-41ed-bf79-7f37c50ac31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:22.914835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 468/914] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:23.184696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e414813-c319-430b-b657-72fd9e2babe3 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:23.273419 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:23.293632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e414813-c319-430b-b657-72fd9e2babe3 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:23.297483 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e6120-1247-426c-b3f2-4e68bd5e6f12 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:23.297942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e6120-1247-426c-b3f2-4e68bd5e6f12 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:23.298017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e6120-1247-426c-b3f2-4e68bd5e6f12 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:23.299816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 469/915] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:23.305134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8481f-3115-44e7-aacd-2f76f3fc8553 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:23.305423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8481f-3115-44e7-aacd-2f76f3fc8553 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:23.305700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8481f-3115-44e7-aacd-2f76f3fc8553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:23.325098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-b2a8481f-3115-44e7-aacd-2f76f3fc8553 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:23.326468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b2a8481f-3115-44e7-aacd-2f76f3fc8553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:23.333279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6359c-75ab-4de8-81c1-901742bf85bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:23.334306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6359c-75ab-4de8-81c1-901742bf85bb None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:23.335297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6359c-75ab-4de8-81c1-901742bf85bb None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:23.335781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6359c-75ab-4de8-81c1-901742bf85bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:23.352444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6359c-75ab-4de8-81c1-901742bf85bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-6cc6359c-75ab-4de8-81c1-901742bf85bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:23.360059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:23.361000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:23.361845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:23.363925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 447/916] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 22 17:06:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:23.369897 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6359c-75ab-4de8-81c1-901742bf85bb None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:23.416273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 470/917] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:23.489264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6359c-75ab-4de8-81c1-901742bf85bb None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:23.527520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc5e3e-c3c3-4318-85d3-661450cb1ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:23.528147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc5e3e-c3c3-4318-85d3-661450cb1ab3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:23.528420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc5e3e-c3c3-4318-85d3-661450cb1ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:23.528687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc5e3e-c3c3-4318-85d3-661450cb1ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:23.529364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc5e3e-c3c3-4318-85d3-661450cb1ab3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:23.529663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc5e3e-c3c3-4318-85d3-661450cb1ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:23.532700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc5e3e-c3c3-4318-85d3-661450cb1ab3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:23.543645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 448/918] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 22 17:06:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:23.550276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8481f-3115-44e7-aacd-2f76f3fc8553 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:23.589539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:23.672026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8481f-3115-44e7-aacd-2f76f3fc8553 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:23.696285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc5e3e-c3c3-4318-85d3-661450cb1ab3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:23.706828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:23.715002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d60a4f-b05f-4623-9a20-b76ce42ea391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:23.715917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d60a4f-b05f-4623-9a20-b76ce42ea391 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:23.716198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d60a4f-b05f-4623-9a20-b76ce42ea391 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:23.716627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d60a4f-b05f-4623-9a20-b76ce42ea391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:23.730891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d60a4f-b05f-4623-9a20-b76ce42ea391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:23.732365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d60a4f-b05f-4623-9a20-b76ce42ea391 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-80d60a4f-b05f-4623-9a20-b76ce42ea391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:23.754694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:23.755678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:23.756576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:23.756576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:23.756576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:23.759585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 471/919] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:23.766659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d60a4f-b05f-4623-9a20-b76ce42ea391 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:23.821997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc5e3e-c3c3-4318-85d3-661450cb1ab3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:23.858900 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:23.861595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fbc3c-e8bb-4906-95a4-1955ad30b91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:23.862303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fbc3c-e8bb-4906-95a4-1955ad30b91d None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:23.862621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fbc3c-e8bb-4906-95a4-1955ad30b91d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:23.862903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fbc3c-e8bb-4906-95a4-1955ad30b91d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:23.863574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73fbc3c-e8bb-4906-95a4-1955ad30b91d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:23.865119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73fbc3c-e8bb-4906-95a4-1955ad30b91d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:23.874822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d60a4f-b05f-4623-9a20-b76ce42ea391 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-a73fbc3c-e8bb-4906-95a4-1955ad30b91d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:23.875963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:23.877118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 17:06:23.878146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:23.882089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 449/920] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:06:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:23.882994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fbc3c-e8bb-4906-95a4-1955ad30b91d None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:23.914393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698d0a8-657c-4937-bc27-466ca5e6e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:23.915286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698d0a8-657c-4937-bc27-466ca5e6e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:23.915630 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698d0a8-657c-4937-bc27-466ca5e6e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:23.916313 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698d0a8-657c-4937-bc27-466ca5e6e9e4 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:23.917256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698d0a8-657c-4937-bc27-466ca5e6e9e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:23.917620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698d0a8-657c-4937-bc27-466ca5e6e9e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:23.919799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698d0a8-657c-4937-bc27-466ca5e6e9e4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:23.933817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 472/921] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:23.940108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698d0a8-657c-4937-bc27-466ca5e6e9e4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:23.976545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:24.012754 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fbc3c-e8bb-4906-95a4-1955ad30b91d None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:24.050585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045948e-9c08-418d-8d01-797c1862906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:24.051196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045948e-9c08-418d-8d01-797c1862906f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.051482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045948e-9c08-418d-8d01-797c1862906f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.051755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045948e-9c08-418d-8d01-797c1862906f None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.052404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045948e-9c08-418d-8d01-797c1862906f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:24.052693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045948e-9c08-418d-8d01-797c1862906f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:24.054135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045948e-9c08-418d-8d01-797c1862906f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:24.064966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 450/922] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:06:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:24.066136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045948e-9c08-418d-8d01-797c1862906f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:24.068131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698d0a8-657c-4937-bc27-466ca5e6e9e4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:24.107269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5828f-77cf-4492-8023-2036415fe7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:24.108271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5828f-77cf-4492-8023-2036415fe7de None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.108792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5828f-77cf-4492-8023-2036415fe7de None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.109276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5828f-77cf-4492-8023-2036415fe7de None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.117487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5828f-77cf-4492-8023-2036415fe7de None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:24.119423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5828f-77cf-4492-8023-2036415fe7de None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-abe5828f-77cf-4492-8023-2036415fe7de None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:24.133724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:24.134767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:24.135752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:24.135752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:24.135752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:24.135752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:24.135752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:24.135752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:24.135752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:24.135752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:24.135752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:24.135752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:24.143676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 473/923] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 22 17:06:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:24.144883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5828f-77cf-4492-8023-2036415fe7de None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:24.155052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:24.175832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045948e-9c08-418d-8d01-797c1862906f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:24.210705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:24.216283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c270e6-9694-4dcb-a229-35bc16bf9100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:24.216894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c270e6-9694-4dcb-a229-35bc16bf9100 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.217173 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c270e6-9694-4dcb-a229-35bc16bf9100 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.217640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c270e6-9694-4dcb-a229-35bc16bf9100 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.218211 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c270e6-9694-4dcb-a229-35bc16bf9100 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:24.218542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c270e6-9694-4dcb-a229-35bc16bf9100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:24.220069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c270e6-9694-4dcb-a229-35bc16bf9100 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:24.225993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 451/924] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 22 17:06:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:24.231782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c270e6-9694-4dcb-a229-35bc16bf9100 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:24.261430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5828f-77cf-4492-8023-2036415fe7de None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:24.299325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d28163-c7bc-4635-a474-a6e598553654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:24.300735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d28163-c7bc-4635-a474-a6e598553654 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.301469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d28163-c7bc-4635-a474-a6e598553654 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.302143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d28163-c7bc-4635-a474-a6e598553654 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.303071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d28163-c7bc-4635-a474-a6e598553654 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:24.303583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d28163-c7bc-4635-a474-a6e598553654 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:24.305356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d28163-c7bc-4635-a474-a6e598553654 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:24.333784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 474/925] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 22 17:06:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:24.339983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d28163-c7bc-4635-a474-a6e598553654 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:24.354406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c270e6-9694-4dcb-a229-35bc16bf9100 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:24.401178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5beccf-568a-40a3-89ea-ac4938873a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:24.401793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5beccf-568a-40a3-89ea-ac4938873a5b None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.402133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5beccf-568a-40a3-89ea-ac4938873a5b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.402982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5beccf-568a-40a3-89ea-ac4938873a5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.403653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5beccf-568a-40a3-89ea-ac4938873a5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:24.407399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5beccf-568a-40a3-89ea-ac4938873a5b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:24.414704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:24.453676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d28163-c7bc-4635-a474-a6e598553654 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:24.473988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 452/926] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:06:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:24.475433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b43f5-69a1-4567-bc59-023c328afb7b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.475815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b43f5-69a1-4567-bc59-023c328afb7b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.476190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b43f5-69a1-4567-bc59-023c328afb7b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.478519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 453/927] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:24.484094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72940133-5583-48ec-b085-5a77fda1d5de None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.484383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72940133-5583-48ec-b085-5a77fda1d5de None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.489399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72940133-5583-48ec-b085-5a77fda1d5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.507145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-72940133-5583-48ec-b085-5a77fda1d5de None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:24.508374 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-72940133-5583-48ec-b085-5a77fda1d5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:24.513576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843d6ae-7875-4d32-ba53-1d941ac2a327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:24.515182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843d6ae-7875-4d32-ba53-1d941ac2a327 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.515959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843d6ae-7875-4d32-ba53-1d941ac2a327 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.516436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843d6ae-7875-4d32-ba53-1d941ac2a327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.532715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843d6ae-7875-4d32-ba53-1d941ac2a327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:24.535698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843d6ae-7875-4d32-ba53-1d941ac2a327 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-0843d6ae-7875-4d32-ba53-1d941ac2a327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:24.558246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:24.559300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:24.560201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:24.560201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:24.560201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:24.568204 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 475/928] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:24.568204 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d75cf0-1e65-4f97-9355-f5cde9b42c67 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.568456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d75cf0-1e65-4f97-9355-f5cde9b42c67 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.569067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d75cf0-1e65-4f97-9355-f5cde9b42c67 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.570629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 476/929] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:24.572144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d75cf0-1e65-4f97-9355-f5cde9b42c67 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:24.572644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:24.596043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 454/930] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:24.604789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e9b54-07e6-49fd-9a51-14c2b8f9bfeb None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.605727 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e9b54-07e6-49fd-9a51-14c2b8f9bfeb None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.606219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e9b54-07e6-49fd-9a51-14c2b8f9bfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.636260 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-af6e9b54-07e6-49fd-9a51-14c2b8f9bfeb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:24.638139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-af6e9b54-07e6-49fd-9a51-14c2b8f9bfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:24.713922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 455/931] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:24.718410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d75cf0-1e65-4f97-9355-f5cde9b42c67 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:24.733765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:24.755068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5621f-185e-4531-95d0-dff87e8ce015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:24.755730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5621f-185e-4531-95d0-dff87e8ce015 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.756015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5621f-185e-4531-95d0-dff87e8ce015 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.756337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5621f-185e-4531-95d0-dff87e8ce015 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.756989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5621f-185e-4531-95d0-dff87e8ce015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:24.757295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5621f-185e-4531-95d0-dff87e8ce015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:24.758685 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5621f-185e-4531-95d0-dff87e8ce015 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:24.771695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 477/932] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:24.777864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e9b54-07e6-49fd-9a51-14c2b8f9bfeb None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:24.867614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5621f-185e-4531-95d0-dff87e8ce015 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:24.887304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e9b54-07e6-49fd-9a51-14c2b8f9bfeb None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:24.903224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:24.932261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd9b8d-d363-43b3-a481-201adb28de19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:24.932899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd9b8d-d363-43b3-a481-201adb28de19 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:24.933176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd9b8d-d363-43b3-a481-201adb28de19 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:24.933448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd9b8d-d363-43b3-a481-201adb28de19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:24.943281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd9b8d-d363-43b3-a481-201adb28de19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:24.945693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd9b8d-d363-43b3-a481-201adb28de19 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-94bd9b8d-d363-43b3-a481-201adb28de19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:24.961446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:24.962401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:24.963474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:24.963474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:24.963474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:24.963474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:24.963474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:24.963474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:24.963474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:24.963474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:24.965607 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 456/933] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:24.972637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd9b8d-d363-43b3-a481-201adb28de19 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:24.993063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5621f-185e-4531-95d0-dff87e8ce015 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:25.031430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5ee81-59a5-4a7d-b18f-f3a6b152d16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:25.032315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5ee81-59a5-4a7d-b18f-f3a6b152d16d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:25.032835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5ee81-59a5-4a7d-b18f-f3a6b152d16d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:25.033296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5ee81-59a5-4a7d-b18f-f3a6b152d16d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:25.034129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5ee81-59a5-4a7d-b18f-f3a6b152d16d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:25.035722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5ee81-59a5-4a7d-b18f-f3a6b152d16d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-71c5ee81-59a5-4a7d-b18f-f3a6b152d16d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:25.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:25.048938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:25.050250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:25.053793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 478/934] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:25.054692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5ee81-59a5-4a7d-b18f-f3a6b152d16d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:25.096118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:25.098778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd9b8d-d363-43b3-a481-201adb28de19 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:25.135697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99248e-6a28-4fb2-947b-104c3daca1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:25.136207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99248e-6a28-4fb2-947b-104c3daca1e1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:25.136478 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99248e-6a28-4fb2-947b-104c3daca1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:25.136743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99248e-6a28-4fb2-947b-104c3daca1e1 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:25.137437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99248e-6a28-4fb2-947b-104c3daca1e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:25.137735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99248e-6a28-4fb2-947b-104c3daca1e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:25.139744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99248e-6a28-4fb2-947b-104c3daca1e1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:25.159919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 457/935] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:25.160652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99248e-6a28-4fb2-947b-104c3daca1e1 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:25.169468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5ee81-59a5-4a7d-b18f-f3a6b152d16d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:25.194664 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:25.210905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93519a23-0a9c-452a-a065-3b96a7eb31f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:25.211713 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93519a23-0a9c-452a-a065-3b96a7eb31f8 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:25.212372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93519a23-0a9c-452a-a065-3b96a7eb31f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:25.212650 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93519a23-0a9c-452a-a065-3b96a7eb31f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:25.213609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93519a23-0a9c-452a-a065-3b96a7eb31f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:25.215453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93519a23-0a9c-452a-a065-3b96a7eb31f8 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-93519a23-0a9c-452a-a065-3b96a7eb31f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:25.227956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:25.229079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 17:06:25.229980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:25.234466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 479/936] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Aug 22 17:06:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:25.235649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93519a23-0a9c-452a-a065-3b96a7eb31f8 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:25.274347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99248e-6a28-4fb2-947b-104c3daca1e1 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:25.312787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83731b9f-b026-4985-b067-e526b4f0b2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:25.313390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83731b9f-b026-4985-b067-e526b4f0b2bd None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:25.313663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83731b9f-b026-4985-b067-e526b4f0b2bd None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:25.313926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83731b9f-b026-4985-b067-e526b4f0b2bd None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:25.314578 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83731b9f-b026-4985-b067-e526b4f0b2bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:25.314877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83731b9f-b026-4985-b067-e526b4f0b2bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:25.316231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83731b9f-b026-4985-b067-e526b4f0b2bd None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:25.326242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 458/937] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 22 17:06:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:25.326242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83731b9f-b026-4985-b067-e526b4f0b2bd None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:25.352204 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:25.355313 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93519a23-0a9c-452a-a065-3b96a7eb31f8 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:25.392276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57360f96-cc69-4899-a40f-efde070d67e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:25.392848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57360f96-cc69-4899-a40f-efde070d67e0 None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:25.393121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57360f96-cc69-4899-a40f-efde070d67e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:25.393389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57360f96-cc69-4899-a40f-efde070d67e0 None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/9001ee719e294f79b35f13389ac9cbaa/roles/ccb62dfdcc3947e9bdb4bba0f88e4e10` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:25.395574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57360f96-cc69-4899-a40f-efde070d67e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=9001ee719e294f79b35f13389ac9cbaa, role_id=ccb62dfdcc3947e9bdb4bba0f88e4e10)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:25.397094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57360f96-cc69-4899-a40f-efde070d67e0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:25.433017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:25.439481 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83731b9f-b026-4985-b067-e526b4f0b2bd None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:25.477878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c2a5c-c9b7-4a79-bb8b-a3f369853eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:25.478508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c2a5c-c9b7-4a79-bb8b-a3f369853eea None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:25.478774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c2a5c-c9b7-4a79-bb8b-a3f369853eea None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:25.479039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c2a5c-c9b7-4a79-bb8b-a3f369853eea None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:25.479659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c2a5c-c9b7-4a79-bb8b-a3f369853eea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:25.479953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c2a5c-c9b7-4a79-bb8b-a3f369853eea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:25.481354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c2a5c-c9b7-4a79-bb8b-a3f369853eea None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:25.490957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 459/938] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:25.490957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c2a5c-c9b7-4a79-bb8b-a3f369853eea None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:25.514751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 480/939] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:06:25 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/9001ee719e294f79b35f13389ac9cbaa/roles/ccb62dfdcc3947e9bdb4bba0f88e4e10 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:06:25.515925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57360f96-cc69-4899-a40f-efde070d67e0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:25.639992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c2a5c-c9b7-4a79-bb8b-a3f369853eea None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:25.667732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57360f96-cc69-4899-a40f-efde070d67e0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:25.691765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b1601-c4cb-4253-aeca-59b005f23f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:25.691765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b1601-c4cb-4253-aeca-59b005f23f1a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:25.692051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b1601-c4cb-4253-aeca-59b005f23f1a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:25.692276 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b1601-c4cb-4253-aeca-59b005f23f1a None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:25.692900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b1601-c4cb-4253-aeca-59b005f23f1a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:25.693199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b1601-c4cb-4253-aeca-59b005f23f1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:25.694588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b1601-c4cb-4253-aeca-59b005f23f1a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:25.700568 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 460/940] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 22 17:06:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:25.706869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b1601-c4cb-4253-aeca-59b005f23f1a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:25.719308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96639e7b-a20f-43bb-ab03-d9b7bd1e7ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:25.720406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96639e7b-a20f-43bb-ab03-d9b7bd1e7ff8 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:25.720935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96639e7b-a20f-43bb-ab03-d9b7bd1e7ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:25.721391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96639e7b-a20f-43bb-ab03-d9b7bd1e7ff8 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:25.722380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96639e7b-a20f-43bb-ab03-d9b7bd1e7ff8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:25.722741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96639e7b-a20f-43bb-ab03-d9b7bd1e7ff8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:25.724165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96639e7b-a20f-43bb-ab03-d9b7bd1e7ff8 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:25.730522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 481/941] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:25.738921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:25.740857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96639e7b-a20f-43bb-ab03-d9b7bd1e7ff8 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:25.772658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:25.823501 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b1601-c4cb-4253-aeca-59b005f23f1a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:25.869053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5e973-04ba-4320-a7a6-746def759dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:25.869720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5e973-04ba-4320-a7a6-746def759dec None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:25.869986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5e973-04ba-4320-a7a6-746def759dec None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:25.870269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5e973-04ba-4320-a7a6-746def759dec None admin] PATH_INFO: `/v3/endpoints` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:25.870990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5e973-04ba-4320-a7a6-746def759dec None admin] RBAC: Adding query filter params (interface=public, service_id=fd00ab9847324cada1a3b154b13c809e, region_id=RegionOne) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:25.871989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5e973-04ba-4320-a7a6-746def759dec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:25.873356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5e973-04ba-4320-a7a6-746def759dec None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:25.880085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 461/942] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 22 17:06:25 2022] GET /identity/v3/endpoints?service_id=fd00ab9847324cada1a3b154b13c809e&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:25.884261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96639e7b-a20f-43bb-ab03-d9b7bd1e7ff8 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:25.885343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5e973-04ba-4320-a7a6-746def759dec None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:25.934090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b03a1-957a-4315-86e0-f1b6cb3d3fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:25.935014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b03a1-957a-4315-86e0-f1b6cb3d3fc2 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:25.935480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b03a1-957a-4315-86e0-f1b6cb3d3fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:25.936019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b03a1-957a-4315-86e0-f1b6cb3d3fc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:25.937067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b03a1-957a-4315-86e0-f1b6cb3d3fc2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:25.938751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b03a1-957a-4315-86e0-f1b6cb3d3fc2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:25.997862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:26.010888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5e973-04ba-4320-a7a6-746def759dec None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:26.047458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0030fab-4cbd-4682-9375-777d0e91df27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:26.048069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0030fab-4cbd-4682-9375-777d0e91df27 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:26.048332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0030fab-4cbd-4682-9375-777d0e91df27 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:26.048597 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0030fab-4cbd-4682-9375-777d0e91df27 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:26.049199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0030fab-4cbd-4682-9375-777d0e91df27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:26.050586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0030fab-4cbd-4682-9375-777d0e91df27 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:26.056598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 462/943] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 22 17:06:25 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:06:26.068712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 482/944] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:06:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:26.641688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbb47b-de8e-4294-bdd9-8fb910bfc6dd None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:26.642017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbb47b-de8e-4294-bdd9-8fb910bfc6dd None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:26.642294 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbb47b-de8e-4294-bdd9-8fb910bfc6dd None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:26.643982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 463/945] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:26.649375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e5eb9-76cd-40bd-9ded-b661a24655e8 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:26.649839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e5eb9-76cd-40bd-9ded-b661a24655e8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:26.649917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e5eb9-76cd-40bd-9ded-b661a24655e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:26.669096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-4c7e5eb9-76cd-40bd-9ded-b661a24655e8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:26.670930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4c7e5eb9-76cd-40bd-9ded-b661a24655e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:26.749187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 483/946] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:27.020996 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbb47b-de8e-4294-bdd9-8fb910bfc6dd None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:27.132236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbb47b-de8e-4294-bdd9-8fb910bfc6dd None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:27.169529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:27.170146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:27.171163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:27.171442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:27.172286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:27.173722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:27.176378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: WARNING keystone.server.flask.common [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 22 17:06:27.224645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.sql.core [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 17:06:27.224645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 22 17:06:27.224645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [parameters: {'id': '44eba556a57441b59fb3606318593c4a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 22 17:06:27.224645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82145) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 44eba556a57441b59fb3606318593c4a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 44eba556a57441b59fb3606318593c4a. Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application conn.query(q) Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 17:06:27.243404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application result.read() Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 22 17:06:27.244290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application next(self.gen) Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application yield resource Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application next(self.gen) Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise self.value Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application session.commit() Aug 22 17:06:27.245849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self.session.flush() Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application compat.raise_( Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 17:06:27.248157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 17:06:27.258209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application conn.query(q) Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application result.read() Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 17:06:27.260171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [parameters: {'id': '44eba556a57441b59fb3606318593c4a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:27.263167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:27.265633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:27.267849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:27.267849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 22 17:06:27.267849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 22 17:06:27.267849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:27.267849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:27.267849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 22 17:06:27.267849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 22 17:06:27.267849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 44eba556a57441b59fb3606318593c4a. Aug 22 17:06:27.267849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:27.267849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 464/947] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Aug 22 17:06:27 2022] POST /identity/v3/projects => generated 262 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:27.269217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e5eb9-76cd-40bd-9ded-b661a24655e8 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:27.269217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:27.373289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e5eb9-76cd-40bd-9ded-b661a24655e8 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:27.412947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dcfcd-623f-43ec-bca0-c1ba9d1ffd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:27.413995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dcfcd-623f-43ec-bca0-c1ba9d1ffd40 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:27.414543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dcfcd-623f-43ec-bca0-c1ba9d1ffd40 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:27.415009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dcfcd-623f-43ec-bca0-c1ba9d1ffd40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:27.423548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dcfcd-623f-43ec-bca0-c1ba9d1ffd40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:27.425387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dcfcd-623f-43ec-bca0-c1ba9d1ffd40 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-8e0dcfcd-623f-43ec-bca0-c1ba9d1ffd40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:27.439136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:27.440071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:27.440991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:27.440991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:27.440991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:27.440991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:27.440991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:27.440991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:27.440991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:27.440991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:27.443703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 484/948] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:27.454571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:27.463263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:27.472015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0b3be-4c10-41cf-8a2b-36122eb9019d None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:27.472300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0b3be-4c10-41cf-8a2b-36122eb9019d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:27.472564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0b3be-4c10-41cf-8a2b-36122eb9019d None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:27.475946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 485/949] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:27.479277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52164d2d-234b-4034-bc64-cbd568565b57 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:27.479560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52164d2d-234b-4034-bc64-cbd568565b57 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:27.479835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52164d2d-234b-4034-bc64-cbd568565b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:27.502429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-52164d2d-234b-4034-bc64-cbd568565b57 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:27.504917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-52164d2d-234b-4034-bc64-cbd568565b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:27.582067 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14785825-e2eb-40d5-85f3-ef06f3204c37 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:27.588247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:27.599355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 486/950] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:27.622996 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a393abb-0ccf-4367-bddf-2c00f7fdebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:27.624527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a393abb-0ccf-4367-bddf-2c00f7fdebab None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:27.625030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a393abb-0ccf-4367-bddf-2c00f7fdebab None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:27.625487 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a393abb-0ccf-4367-bddf-2c00f7fdebab None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:27.626313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a393abb-0ccf-4367-bddf-2c00f7fdebab None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:27.626782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a393abb-0ccf-4367-bddf-2c00f7fdebab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:27.629315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a393abb-0ccf-4367-bddf-2c00f7fdebab None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:27.652374 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 465/951] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:27.879443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52164d2d-234b-4034-bc64-cbd568565b57 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:27.984079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52164d2d-234b-4034-bc64-cbd568565b57 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:28.019959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d477b-1435-4050-85c0-adf7dab99150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:28.020606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d477b-1435-4050-85c0-adf7dab99150 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.020882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d477b-1435-4050-85c0-adf7dab99150 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.021153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d477b-1435-4050-85c0-adf7dab99150 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.032900 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d477b-1435-4050-85c0-adf7dab99150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-772d477b-1435-4050-85c0-adf7dab99150 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:28.037588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:28.038595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:28.039477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 487/952] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 22 17:06:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:28.045810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a393abb-0ccf-4367-bddf-2c00f7fdebab None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:28.058543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50093b-5e41-4b7f-a503-dd5f99c4294f None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.058543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50093b-5e41-4b7f-a503-dd5f99c4294f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.058818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50093b-5e41-4b7f-a503-dd5f99c4294f None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.060417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 488/953] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:28.066150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a02d2a-fa12-4bf8-a202-5f2b4997db79 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.066432 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a02d2a-fa12-4bf8-a202-5f2b4997db79 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.066737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a02d2a-fa12-4bf8-a202-5f2b4997db79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.085274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-98a02d2a-fa12-4bf8-a202-5f2b4997db79 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:28.086463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-98a02d2a-fa12-4bf8-a202-5f2b4997db79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:28.130983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:28.151043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:28.167828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a393abb-0ccf-4367-bddf-2c00f7fdebab None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:28.169793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 489/954] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:28.178570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ed157-0953-4728-b247-04c010b5d7d3 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.178570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ed157-0953-4728-b247-04c010b5d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.178570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ed157-0953-4728-b247-04c010b5d7d3 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.178570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 490/955] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:28.180350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945054e8-79ad-4e00-a4cb-26fbba20eed0 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.180788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945054e8-79ad-4e00-a4cb-26fbba20eed0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.181221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945054e8-79ad-4e00-a4cb-26fbba20eed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.200012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-945054e8-79ad-4e00-a4cb-26fbba20eed0 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:28.202030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-945054e8-79ad-4e00-a4cb-26fbba20eed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:28.207896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebc7d2-022e-48bf-9687-9c12ed119360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:28.208518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebc7d2-022e-48bf-9687-9c12ed119360 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.208801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebc7d2-022e-48bf-9687-9c12ed119360 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.209080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebc7d2-022e-48bf-9687-9c12ed119360 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.209758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebc7d2-022e-48bf-9687-9c12ed119360 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:28.210067 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebc7d2-022e-48bf-9687-9c12ed119360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:28.211535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebc7d2-022e-48bf-9687-9c12ed119360 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:28.220657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 466/956] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 22 17:06:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:28.227755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebc7d2-022e-48bf-9687-9c12ed119360 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:28.282630 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 491/957] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:28.351495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebc7d2-022e-48bf-9687-9c12ed119360 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:28.381273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:28.387041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93865d27-806a-4fd9-ad34-6cdd3f158504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:28.389103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93865d27-806a-4fd9-ad34-6cdd3f158504 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.389103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93865d27-806a-4fd9-ad34-6cdd3f158504 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.389103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93865d27-806a-4fd9-ad34-6cdd3f158504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.399106 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93865d27-806a-4fd9-ad34-6cdd3f158504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:28.400608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93865d27-806a-4fd9-ad34-6cdd3f158504 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-93865d27-806a-4fd9-ad34-6cdd3f158504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:28.414814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:28.415762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:28.416694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:28.416694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:28.416694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:28.419182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 467/958] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:28.425563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945054e8-79ad-4e00-a4cb-26fbba20eed0 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:28.442538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93865d27-806a-4fd9-ad34-6cdd3f158504 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:28.547419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945054e8-79ad-4e00-a4cb-26fbba20eed0 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:28.552399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93865d27-806a-4fd9-ad34-6cdd3f158504 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:28.566808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:28.586613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269338a-4c8e-4775-b3c4-5d92b40ba004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:28.587106 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857a913-3d43-4a6e-b3df-7af44e884a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:28.587226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269338a-4c8e-4775-b3c4-5d92b40ba004 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.587506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269338a-4c8e-4775-b3c4-5d92b40ba004 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.587737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857a913-3d43-4a6e-b3df-7af44e884a68 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.587847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269338a-4c8e-4775-b3c4-5d92b40ba004 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.588012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857a913-3d43-4a6e-b3df-7af44e884a68 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.588278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857a913-3d43-4a6e-b3df-7af44e884a68 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.588436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269338a-4c8e-4775-b3c4-5d92b40ba004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:28.588821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269338a-4c8e-4775-b3c4-5d92b40ba004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:28.588922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857a913-3d43-4a6e-b3df-7af44e884a68 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:28.590201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857a913-3d43-4a6e-b3df-7af44e884a68 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:28.590295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269338a-4c8e-4775-b3c4-5d92b40ba004 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-0857a913-3d43-4a6e-b3df-7af44e884a68 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:28.600103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:28.601213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:28.602083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 468/959] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:28.602926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857a913-3d43-4a6e-b3df-7af44e884a68 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:28.610611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 492/960] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:28.610611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269338a-4c8e-4775-b3c4-5d92b40ba004 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:28.661184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:28.720152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269338a-4c8e-4775-b3c4-5d92b40ba004 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:28.725281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857a913-3d43-4a6e-b3df-7af44e884a68 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:28.762686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc714e79-c232-4c26-95c5-cde281574a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:28.762924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346386f-7c06-46d0-91e2-90a49337eefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:28.763416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346386f-7c06-46d0-91e2-90a49337eefa None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.763695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346386f-7c06-46d0-91e2-90a49337eefa None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.764007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346386f-7c06-46d0-91e2-90a49337eefa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.764616 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346386f-7c06-46d0-91e2-90a49337eefa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:28.764849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc714e79-c232-4c26-95c5-cde281574a9b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.765315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc714e79-c232-4c26-95c5-cde281574a9b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.765778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc714e79-c232-4c26-95c5-cde281574a9b None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.766016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346386f-7c06-46d0-91e2-90a49337eefa None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:28.766853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc714e79-c232-4c26-95c5-cde281574a9b None admin] RBAC: Adding query filter params (name=compute) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:28.767304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc714e79-c232-4c26-95c5-cde281574a9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:28.769482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc714e79-c232-4c26-95c5-cde281574a9b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:28.774909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 493/961] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-1346386f-7c06-46d0-91e2-90a49337eefa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:28.775124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:28.776084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:28.776926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 469/962] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:28.778580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc714e79-c232-4c26-95c5-cde281574a9b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:28.781242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346386f-7c06-46d0-91e2-90a49337eefa None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:28.825419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:28.892558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc714e79-c232-4c26-95c5-cde281574a9b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:28.898409 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346386f-7c06-46d0-91e2-90a49337eefa None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:28.911542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:28.928469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a2371-ef9e-465d-8d04-075e3208c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:28.929103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a2371-ef9e-465d-8d04-075e3208c541 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.929384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a2371-ef9e-465d-8d04-075e3208c541 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.929657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a2371-ef9e-465d-8d04-075e3208c541 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.937109 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6ee9f-5b1e-4b57-aa4e-8f4cb8b457d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:28.937732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a2371-ef9e-465d-8d04-075e3208c541 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:28.938078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6ee9f-5b1e-4b57-aa4e-8f4cb8b457d8 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:28.938547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6ee9f-5b1e-4b57-aa4e-8f4cb8b457d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:28.938959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6ee9f-5b1e-4b57-aa4e-8f4cb8b457d8 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:28.939242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a2371-ef9e-465d-8d04-075e3208c541 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:28.940068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6ee9f-5b1e-4b57-aa4e-8f4cb8b457d8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:28.940515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6ee9f-5b1e-4b57-aa4e-8f4cb8b457d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:28.942267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6ee9f-5b1e-4b57-aa4e-8f4cb8b457d8 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:28.947398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 470/963] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:28.948466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6ee9f-5b1e-4b57-aa4e-8f4cb8b457d8 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-432a2371-ef9e-465d-8d04-075e3208c541 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:28.950760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:28.952340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:28.953608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:28.953608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:28.953608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:28.953608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:28.953608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:28.953608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:28.953608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:28.953608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:28.953608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 17:06:28.953608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:28.957652 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 494/964] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 22 17:06:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:28.958766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a2371-ef9e-465d-8d04-075e3208c541 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:29.081439 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:29.084887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a2371-ef9e-465d-8d04-075e3208c541 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:29.098418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6ee9f-5b1e-4b57-aa4e-8f4cb8b457d8 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:29.126357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd98730-080d-4330-b1aa-a2d050f7bdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:29.127477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd98730-080d-4330-b1aa-a2d050f7bdb8 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:29.129129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd98730-080d-4330-b1aa-a2d050f7bdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:29.129129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd98730-080d-4330-b1aa-a2d050f7bdb8 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:29.130166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd98730-080d-4330-b1aa-a2d050f7bdb8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:29.130166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd98730-080d-4330-b1aa-a2d050f7bdb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:29.132247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd98730-080d-4330-b1aa-a2d050f7bdb8 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:29.147258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168780ab-258f-4ef0-a018-706b822406e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:29.148304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168780ab-258f-4ef0-a018-706b822406e6 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:29.148761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168780ab-258f-4ef0-a018-706b822406e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:29.149336 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168780ab-258f-4ef0-a018-706b822406e6 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:29.149982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168780ab-258f-4ef0-a018-706b822406e6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:29.150287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168780ab-258f-4ef0-a018-706b822406e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:29.151837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168780ab-258f-4ef0-a018-706b822406e6 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:29.158574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 495/965] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 22 17:06:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:29.160265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd98730-080d-4330-b1aa-a2d050f7bdb8 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:29.162457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 471/966] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 22 17:06:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:29.168698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168780ab-258f-4ef0-a018-706b822406e6 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:29.218940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:29.279027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd98730-080d-4330-b1aa-a2d050f7bdb8 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:29.286017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168780ab-258f-4ef0-a018-706b822406e6 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:29.319169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8574f-5a30-4ad0-b6b5-b539a862392f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:29.320282 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8574f-5a30-4ad0-b6b5-b539a862392f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:29.320779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8574f-5a30-4ad0-b6b5-b539a862392f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:29.321254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8574f-5a30-4ad0-b6b5-b539a862392f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:29.325142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619229d0-f764-43a6-87df-0608c64e7039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:29.326497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619229d0-f764-43a6-87df-0608c64e7039 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:29.326497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619229d0-f764-43a6-87df-0608c64e7039 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:29.326497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619229d0-f764-43a6-87df-0608c64e7039 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:29.327112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619229d0-f764-43a6-87df-0608c64e7039 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:29.327419 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619229d0-f764-43a6-87df-0608c64e7039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:29.328880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619229d0-f764-43a6-87df-0608c64e7039 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:29.339790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 472/967] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 22 17:06:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:29.341333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619229d0-f764-43a6-87df-0608c64e7039 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:29.354330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:29.402300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8574f-5a30-4ad0-b6b5-b539a862392f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:29.404234 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8574f-5a30-4ad0-b6b5-b539a862392f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:29.457947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619229d0-f764-43a6-87df-0608c64e7039 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:29.470934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:29.499313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138cb81-74fd-43d1-8ca8-11fb19a0773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:29.500099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138cb81-74fd-43d1-8ca8-11fb19a0773a None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:29.500948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138cb81-74fd-43d1-8ca8-11fb19a0773a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:29.500948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138cb81-74fd-43d1-8ca8-11fb19a0773a None admin] PATH_INFO: `/v3/endpoints` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:29.501899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138cb81-74fd-43d1-8ca8-11fb19a0773a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:29.503621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138cb81-74fd-43d1-8ca8-11fb19a0773a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:29.533442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 496/968] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:29 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 17:06:29.534725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8574f-5a30-4ad0-b6b5-b539a862392f None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:29.605262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:29.606544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 473/969] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:06:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:29.661750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8574f-5a30-4ad0-b6b5-b539a862392f None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:29.710852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3009d2-ed5f-4eb4-92a8-4a7eaf7d212b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:29.711035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3009d2-ed5f-4eb4-92a8-4a7eaf7d212b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:29.711305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3009d2-ed5f-4eb4-92a8-4a7eaf7d212b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:29.711622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d252b88-b6cd-42ef-b058-038a23b03d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:29.712385 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d252b88-b6cd-42ef-b058-038a23b03d53 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:29.712761 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d252b88-b6cd-42ef-b058-038a23b03d53 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:29.712926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 474/970] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:29.713239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d252b88-b6cd-42ef-b058-038a23b03d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:29.719352 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90498d07-e8a1-404a-bd35-d40bda2d0739 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:29.719632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90498d07-e8a1-404a-bd35-d40bda2d0739 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:29.719907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90498d07-e8a1-404a-bd35-d40bda2d0739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:29.733039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d252b88-b6cd-42ef-b058-038a23b03d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:29.735772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d252b88-b6cd-42ef-b058-038a23b03d53 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:29.740489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-90498d07-e8a1-404a-bd35-d40bda2d0739 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:29.742231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-90498d07-e8a1-404a-bd35-d40bda2d0739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-2d252b88-b6cd-42ef-b058-038a23b03d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:29.757784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:29.759120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:29.760253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:29.760253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:29.760253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:29.760253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 497/971] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:29.765144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d252b88-b6cd-42ef-b058-038a23b03d53 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:29.844121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 475/972] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:29.882330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d252b88-b6cd-42ef-b058-038a23b03d53 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:29.890454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:29.918818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa422887-6bb1-4122-981c-3b0f5a8d4d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:29.919559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa422887-6bb1-4122-981c-3b0f5a8d4d2a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:29.919855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa422887-6bb1-4122-981c-3b0f5a8d4d2a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:29.920221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa422887-6bb1-4122-981c-3b0f5a8d4d2a None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:29.920987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa422887-6bb1-4122-981c-3b0f5a8d4d2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:29.921347 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa422887-6bb1-4122-981c-3b0f5a8d4d2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:29.922863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa422887-6bb1-4122-981c-3b0f5a8d4d2a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:29.936240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 498/973] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:29.941584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90498d07-e8a1-404a-bd35-d40bda2d0739 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:30.039661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:30.052247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90498d07-e8a1-404a-bd35-d40bda2d0739 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:30.089522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36beec76-2473-4ce8-a104-aec86842823b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:30.090164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beec76-2473-4ce8-a104-aec86842823b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:30.090448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beec76-2473-4ce8-a104-aec86842823b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:30.090757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beec76-2473-4ce8-a104-aec86842823b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:30.098262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36beec76-2473-4ce8-a104-aec86842823b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:30.099782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36beec76-2473-4ce8-a104-aec86842823b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-36beec76-2473-4ce8-a104-aec86842823b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:30.112127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:30.113293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:30.114458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:30.114458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:30.114458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:30.114458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:30.114458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:30.114458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:30.114458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:30.114458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:30.114458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 476/974] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:30.120004 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa422887-6bb1-4122-981c-3b0f5a8d4d2a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:30.122560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36beec76-2473-4ce8-a104-aec86842823b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:30.246172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa422887-6bb1-4122-981c-3b0f5a8d4d2a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:30.250748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36beec76-2473-4ce8-a104-aec86842823b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:30.287339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46605c3-9f67-4203-9b53-e364b1c59796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:30.288323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46605c3-9f67-4203-9b53-e364b1c59796 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:30.288760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46605c3-9f67-4203-9b53-e364b1c59796 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:30.289344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46605c3-9f67-4203-9b53-e364b1c59796 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:30.290434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46605c3-9f67-4203-9b53-e364b1c59796 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:30.290924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46605c3-9f67-4203-9b53-e364b1c59796 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:30.292180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:30.292815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:30.293086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46605c3-9f67-4203-9b53-e364b1c59796 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:30.293200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:30.293635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:30.294270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:30.296732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:30.299995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WARNING keystone.server.flask.common [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 22 17:06:30.309811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 477/975] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:30.318596 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46605c3-9f67-4203-9b53-e364b1c59796 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:30.330273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:30.363099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.sql.core [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 17:06:30.363099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 22 17:06:30.363099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [parameters: {'user_id': '5c0d83d3388948889b6a4ac929a4d1fd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 22 17:06:30.363099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82146) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application conn.query(q) Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application result.read() Aug 22 17:06:30.370656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 17:06:30.375476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application next(self.gen) Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application yield resource Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application next(self.gen) Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise self.value Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application session.commit() Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 17:06:30.376472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self.session.flush() Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application compat.raise_( Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 17:06:30.377387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 17:06:30.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application conn.query(q) Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application result.read() Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 17:06:30.379976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [parameters: {'user_id': '5c0d83d3388948889b6a4ac929a4d1fd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:30.382083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 22 17:06:30.384129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 22 17:06:30.386319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 22 17:06:30.386319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 22 17:06:30.386319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 17:06:30.386319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 17:06:30.386319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 22 17:06:30.386319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:30.386319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 499/976] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 22 17:06:30 2022] POST /identity/v3/users => generated 160 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:30.386319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:30.420936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:30.464178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46605c3-9f67-4203-9b53-e364b1c59796 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:30.501599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1671b-ec1f-4e3c-8b27-13c6204a0fb1 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:30.512186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52307725-67a1-4e20-a97d-481e17c99a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:30.512801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52307725-67a1-4e20-a97d-481e17c99a95 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:30.513078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52307725-67a1-4e20-a97d-481e17c99a95 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:30.513347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52307725-67a1-4e20-a97d-481e17c99a95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:30.514123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52307725-67a1-4e20-a97d-481e17c99a95 None admin] RBAC: Adding query filter params (role.id=ccb62dfdcc3947e9bdb4bba0f88e4e10, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=9001ee719e294f79b35f13389ac9cbaa) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:30.514424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52307725-67a1-4e20-a97d-481e17c99a95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:30.516760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52307725-67a1-4e20-a97d-481e17c99a95 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:30.530509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 478/977] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:30 2022] GET /identity/v3/role_assignments?user.id=9001ee719e294f79b35f13389ac9cbaa&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=ccb62dfdcc3947e9bdb4bba0f88e4e10 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:30.544331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840372c8-655b-4768-96b8-bf105039148d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:30.545663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840372c8-655b-4768-96b8-bf105039148d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:30.545978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840372c8-655b-4768-96b8-bf105039148d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:30.546227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840372c8-655b-4768-96b8-bf105039148d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:30.555104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840372c8-655b-4768-96b8-bf105039148d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:30.556862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840372c8-655b-4768-96b8-bf105039148d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-840372c8-655b-4768-96b8-bf105039148d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:30.571778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:30.572808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:30.573763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 500/978] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:06:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:30.579242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52307725-67a1-4e20-a97d-481e17c99a95 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:30.663668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:30.698172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52307725-67a1-4e20-a97d-481e17c99a95 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:30.741800 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de96db8-54b3-4e7c-8ddf-fdd3c568ee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:30.742420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de96db8-54b3-4e7c-8ddf-fdd3c568ee55 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:30.742750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de96db8-54b3-4e7c-8ddf-fdd3c568ee55 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:30.743024 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de96db8-54b3-4e7c-8ddf-fdd3c568ee55 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:30.743752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de96db8-54b3-4e7c-8ddf-fdd3c568ee55 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:30.744064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de96db8-54b3-4e7c-8ddf-fdd3c568ee55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:30.745558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de96db8-54b3-4e7c-8ddf-fdd3c568ee55 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:30.785707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 479/979] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 22 17:06:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:31.003815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d34ff2-da9c-4395-8079-6f18956f58ea None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:31.004342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d34ff2-da9c-4395-8079-6f18956f58ea None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:31.004342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d34ff2-da9c-4395-8079-6f18956f58ea None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:31.009233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 501/980] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:31.015018 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799c5b8-22a1-40c7-adca-e7957c7bfb14 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:31.015535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799c5b8-22a1-40c7-adca-e7957c7bfb14 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:31.015967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799c5b8-22a1-40c7-adca-e7957c7bfb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:31.035714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3799c5b8-22a1-40c7-adca-e7957c7bfb14 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:31.037203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3799c5b8-22a1-40c7-adca-e7957c7bfb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:31.112327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 480/981] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:31.390417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d34ff2-da9c-4395-8079-6f18956f58ea None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:31.485329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:31.502259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d34ff2-da9c-4395-8079-6f18956f58ea None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:31.539496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb7a0a-7403-43f2-a1c1-cd5f82b63b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:31.540104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb7a0a-7403-43f2-a1c1-cd5f82b63b3f None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:31.540484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb7a0a-7403-43f2-a1c1-cd5f82b63b3f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:31.540682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb7a0a-7403-43f2-a1c1-cd5f82b63b3f None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:31.541294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb7a0a-7403-43f2-a1c1-cd5f82b63b3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:31.542815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb7a0a-7403-43f2-a1c1-cd5f82b63b3f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:31.632194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 502/982] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:06:31 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:31.634096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e4bbf-4d64-428b-9626-6231309b0509 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:31.634581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e4bbf-4d64-428b-9626-6231309b0509 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:31.634862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e4bbf-4d64-428b-9626-6231309b0509 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:31.636744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 481/983] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:31.643596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681eb03-a283-45f0-afc0-0954ba918187 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:31.644208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681eb03-a283-45f0-afc0-0954ba918187 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:31.644657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681eb03-a283-45f0-afc0-0954ba918187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:31.666053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0681eb03-a283-45f0-afc0-0954ba918187 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:31.668072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0681eb03-a283-45f0-afc0-0954ba918187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:31.757516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 503/984] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:32.036647 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e4bbf-4d64-428b-9626-6231309b0509 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:32.105331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:32.147636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e4bbf-4d64-428b-9626-6231309b0509 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:32.183399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b667f6-fde2-4b40-bcbb-a3d6ca2fbf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:32.183956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b667f6-fde2-4b40-bcbb-a3d6ca2fbf01 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:32.184229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b667f6-fde2-4b40-bcbb-a3d6ca2fbf01 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:32.184492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b667f6-fde2-4b40-bcbb-a3d6ca2fbf01 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:32.185072 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b667f6-fde2-4b40-bcbb-a3d6ca2fbf01 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:32.187331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b667f6-fde2-4b40-bcbb-a3d6ca2fbf01 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-23b667f6-fde2-4b40-bcbb-a3d6ca2fbf01 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:32.196060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:32.197092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:32.197948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 482/985] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:32.203907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681eb03-a283-45f0-afc0-0954ba918187 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:32.282128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:32.308645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681eb03-a283-45f0-afc0-0954ba918187 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:32.345992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d6ff3-403e-432e-80cc-d74808d98b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:32.346667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d6ff3-403e-432e-80cc-d74808d98b89 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:32.346948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d6ff3-403e-432e-80cc-d74808d98b89 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:32.347275 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d6ff3-403e-432e-80cc-d74808d98b89 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:32.347917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d6ff3-403e-432e-80cc-d74808d98b89 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:32.348279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d6ff3-403e-432e-80cc-d74808d98b89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:32.349679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d6ff3-403e-432e-80cc-d74808d98b89 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:32.359374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 504/986] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 22 17:06:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:32.360584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b667f6-fde2-4b40-bcbb-a3d6ca2fbf01 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:32.466451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b667f6-fde2-4b40-bcbb-a3d6ca2fbf01 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:32.490537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:32.505853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe39f2e-55f3-4edb-8562-f3b9f41d87c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:32.506463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe39f2e-55f3-4edb-8562-f3b9f41d87c1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:32.506781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe39f2e-55f3-4edb-8562-f3b9f41d87c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:32.507050 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe39f2e-55f3-4edb-8562-f3b9f41d87c1 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:32.507682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe39f2e-55f3-4edb-8562-f3b9f41d87c1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:32.507985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe39f2e-55f3-4edb-8562-f3b9f41d87c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:32.509356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe39f2e-55f3-4edb-8562-f3b9f41d87c1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:32.519867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 483/987] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 22 17:06:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:32.521536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d6ff3-403e-432e-80cc-d74808d98b89 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:32.557101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d481b5-7f6f-470a-8d77-7f7b3518ba2e None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:32.557444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d481b5-7f6f-470a-8d77-7f7b3518ba2e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:32.557652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d481b5-7f6f-470a-8d77-7f7b3518ba2e None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:32.559235 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 484/988] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:32.564823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac8865-c3d4-4d42-82dd-ef694ac4aef3 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:32.565096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac8865-c3d4-4d42-82dd-ef694ac4aef3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:32.565368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac8865-c3d4-4d42-82dd-ef694ac4aef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:32.583623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-8fac8865-c3d4-4d42-82dd-ef694ac4aef3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:32.584845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8fac8865-c3d4-4d42-82dd-ef694ac4aef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:32.637991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d6ff3-403e-432e-80cc-d74808d98b89 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:32.664385 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 485/989] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:32.676364 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b87d2-8141-4653-b025-1f07ea244bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:32.676982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b87d2-8141-4653-b025-1f07ea244bc2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:32.677261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b87d2-8141-4653-b025-1f07ea244bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:32.677534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b87d2-8141-4653-b025-1f07ea244bc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:32.678244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b87d2-8141-4653-b025-1f07ea244bc2 None admin] RBAC: Adding query filter params (interface=admin, service_id=fd00ab9847324cada1a3b154b13c809e, region_id=RegionOne) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:32.678582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b87d2-8141-4653-b025-1f07ea244bc2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:32.679983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b87d2-8141-4653-b025-1f07ea244bc2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:32.691015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 505/990] 10.222.0.20 () {56 vars in 1249 bytes} [Mon Aug 22 17:06:32 2022] GET /identity/v3/endpoints?service_id=fd00ab9847324cada1a3b154b13c809e&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:32.692695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac8865-c3d4-4d42-82dd-ef694ac4aef3 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:32.802284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac8865-c3d4-4d42-82dd-ef694ac4aef3 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:32.820950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:32.838921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4416a6-b59f-461b-88aa-f2fb21ea617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:32.839706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4416a6-b59f-461b-88aa-f2fb21ea617e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:32.839967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4416a6-b59f-461b-88aa-f2fb21ea617e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:32.840310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4416a6-b59f-461b-88aa-f2fb21ea617e None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:32.840985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4416a6-b59f-461b-88aa-f2fb21ea617e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:32.842590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4416a6-b59f-461b-88aa-f2fb21ea617e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:32.858843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 486/991] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 22 17:06:32 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:06:32.941607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b87d2-8141-4653-b025-1f07ea244bc2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:32.944609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:33.051817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b87d2-8141-4653-b025-1f07ea244bc2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:33.086389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57503a0-8570-4064-8d75-7fe0c556fe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:33.086958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57503a0-8570-4064-8d75-7fe0c556fe7f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:33.087232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57503a0-8570-4064-8d75-7fe0c556fe7f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:33.087498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57503a0-8570-4064-8d75-7fe0c556fe7f None admin] PATH_INFO: `/v3/services/image` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:33.088075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57503a0-8570-4064-8d75-7fe0c556fe7f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:33.089434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57503a0-8570-4064-8d75-7fe0c556fe7f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-a57503a0-8570-4064-8d75-7fe0c556fe7f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:33.098972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:33.100169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:33.101231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 506/992] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Aug 22 17:06:32 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:33.109051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4416a6-b59f-461b-88aa-f2fb21ea617e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:33.215848 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4416a6-b59f-461b-88aa-f2fb21ea617e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:33.252394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5683c67-0d5e-4c94-b3f1-40695d16626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:33.253006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5683c67-0d5e-4c94-b3f1-40695d16626f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:33.253279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5683c67-0d5e-4c94-b3f1-40695d16626f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:33.253542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5683c67-0d5e-4c94-b3f1-40695d16626f None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:33.254240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5683c67-0d5e-4c94-b3f1-40695d16626f None admin] RBAC: Adding query filter params (name=image) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:33.254636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5683c67-0d5e-4c94-b3f1-40695d16626f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:33.256057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5683c67-0d5e-4c94-b3f1-40695d16626f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:33.267009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 487/993] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:33.268020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57503a0-8570-4064-8d75-7fe0c556fe7f None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:33.352084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:33.382002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57503a0-8570-4064-8d75-7fe0c556fe7f None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:33.416214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b1684-4d40-4ec4-afd6-67b538522bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:33.416824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b1684-4d40-4ec4-afd6-67b538522bcf None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:33.417115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b1684-4d40-4ec4-afd6-67b538522bcf None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:33.417367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b1684-4d40-4ec4-afd6-67b538522bcf None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:33.417994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b1684-4d40-4ec4-afd6-67b538522bcf None admin] RBAC: Adding query filter params (type=image) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:33.418576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b1684-4d40-4ec4-afd6-67b538522bcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:33.419987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b1684-4d40-4ec4-afd6-67b538522bcf None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:33.430671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 507/994] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:33 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:33.573746 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d552-3170-4305-a071-cc7147e363ff None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:33.574036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d552-3170-4305-a071-cc7147e363ff None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:33.574308 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d552-3170-4305-a071-cc7147e363ff None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:33.575955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 488/995] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:33.581668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960038a-e4f5-4e2d-ad7e-d001e1dba373 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:33.581953 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960038a-e4f5-4e2d-ad7e-d001e1dba373 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:33.582233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960038a-e4f5-4e2d-ad7e-d001e1dba373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:33.601712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-5960038a-e4f5-4e2d-ad7e-d001e1dba373 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:33.602996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5960038a-e4f5-4e2d-ad7e-d001e1dba373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:33.675587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 508/996] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:33.945539 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d552-3170-4305-a071-cc7147e363ff None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:33.998212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:34.056273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d552-3170-4305-a071-cc7147e363ff None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:34.091425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190371a5-7e83-4932-90e6-9d63f843d963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:34.092861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190371a5-7e83-4932-90e6-9d63f843d963 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:34.093282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190371a5-7e83-4932-90e6-9d63f843d963 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:34.093694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190371a5-7e83-4932-90e6-9d63f843d963 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:34.094441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190371a5-7e83-4932-90e6-9d63f843d963 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:34.095974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190371a5-7e83-4932-90e6-9d63f843d963 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-190371a5-7e83-4932-90e6-9d63f843d963 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:34.104665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:34.105813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 17:06:34.106853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:34.113791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 489/997] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:34.114972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960038a-e4f5-4e2d-ad7e-d001e1dba373 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:34.179140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:34.221965 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960038a-e4f5-4e2d-ad7e-d001e1dba373 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:34.256152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b6477-f647-48a3-be72-7229b75b10ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:34.256814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b6477-f647-48a3-be72-7229b75b10ad None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:34.257099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b6477-f647-48a3-be72-7229b75b10ad None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:34.257369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b6477-f647-48a3-be72-7229b75b10ad None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:34.257994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b6477-f647-48a3-be72-7229b75b10ad None admin] RBAC: Adding query filter params (name=compute) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:34.258318 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b6477-f647-48a3-be72-7229b75b10ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:34.259715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b6477-f647-48a3-be72-7229b75b10ad None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:34.270079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 509/998] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:34.271237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190371a5-7e83-4932-90e6-9d63f843d963 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:34.379003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190371a5-7e83-4932-90e6-9d63f843d963 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:34.391089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:34.416224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f1734-a565-465c-af68-1cd532e9452d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:34.416825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1734-a565-465c-af68-1cd532e9452d None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:34.417091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1734-a565-465c-af68-1cd532e9452d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:34.417360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1734-a565-465c-af68-1cd532e9452d None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:34.417993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f1734-a565-465c-af68-1cd532e9452d None admin] RBAC: Adding query filter params (type=compute) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:34.418584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f1734-a565-465c-af68-1cd532e9452d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:34.420019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f1734-a565-465c-af68-1cd532e9452d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:34.430100 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 490/999] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:34.431429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b6477-f647-48a3-be72-7229b75b10ad None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:34.548886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b6477-f647-48a3-be72-7229b75b10ad None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:34.574320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:34.585732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec1c23-fea8-4828-9733-4b041f917826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:34.586361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec1c23-fea8-4828-9733-4b041f917826 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:34.586655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec1c23-fea8-4828-9733-4b041f917826 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:34.586926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec1c23-fea8-4828-9733-4b041f917826 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:34.587624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec1c23-fea8-4828-9733-4b041f917826 None admin] RBAC: Adding query filter params (interface=public, service_id=673861475b444f13806b58a61a200698, region_id=RegionOne) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:34.588508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec1c23-fea8-4828-9733-4b041f917826 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:34.589732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec1c23-fea8-4828-9733-4b041f917826 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:34.599290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 510/1000] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 22 17:06:34 2022] GET /identity/v3/endpoints?service_id=673861475b444f13806b58a61a200698&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:34.600350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f1734-a565-465c-af68-1cd532e9452d None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:34.706839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f1734-a565-465c-af68-1cd532e9452d None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:34.740608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85a4a4-c3af-470c-9f2a-fa36789dcd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:34.741209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a4a4-c3af-470c-9f2a-fa36789dcd59 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:34.741484 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a4a4-c3af-470c-9f2a-fa36789dcd59 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:34.741752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a4a4-c3af-470c-9f2a-fa36789dcd59 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:34.742354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85a4a4-c3af-470c-9f2a-fa36789dcd59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:34.743820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85a4a4-c3af-470c-9f2a-fa36789dcd59 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:34.754718 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 491/1001] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 22 17:06:34 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:06:34.822984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b87fb3-f914-4b3f-b0b3-09df6dc037db None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:34.823233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b87fb3-f914-4b3f-b0b3-09df6dc037db None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:34.823464 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b87fb3-f914-4b3f-b0b3-09df6dc037db None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:34.825009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 511/1002] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:34.831094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30e499-9aa4-466f-ad42-755ec11465d2 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:34.831410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30e499-9aa4-466f-ad42-755ec11465d2 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:34.831651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30e499-9aa4-466f-ad42-755ec11465d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:34.850671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ae30e499-9aa4-466f-ad42-755ec11465d2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:34.851898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ae30e499-9aa4-466f-ad42-755ec11465d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:34.890158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:34.928941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 492/1003] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:35.198269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b87fb3-f914-4b3f-b0b3-09df6dc037db None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:35.305477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b87fb3-f914-4b3f-b0b3-09df6dc037db None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:35.343466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141312b6-8a5a-4e98-8604-c4d135a0a52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:35.343466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141312b6-8a5a-4e98-8604-c4d135a0a52a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:35.343779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141312b6-8a5a-4e98-8604-c4d135a0a52a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:35.343857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141312b6-8a5a-4e98-8604-c4d135a0a52a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:35.345892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141312b6-8a5a-4e98-8604-c4d135a0a52a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:35.345892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141312b6-8a5a-4e98-8604-c4d135a0a52a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-141312b6-8a5a-4e98-8604-c4d135a0a52a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:35.355564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:35.356674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 17:06:35.357763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:35.361778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 512/1004] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 22 17:06:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:35.363197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30e499-9aa4-466f-ad42-755ec11465d2 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:35.390964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55066f0f-b37a-4e2d-a16c-84d35719f021 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:35.391250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55066f0f-b37a-4e2d-a16c-84d35719f021 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:35.391525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55066f0f-b37a-4e2d-a16c-84d35719f021 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:35.392885 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 513/1005] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:35.398164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5d9ba-8b33-41e3-b2a4-620a20afe2c1 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:35.398449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5d9ba-8b33-41e3-b2a4-620a20afe2c1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:35.398755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5d9ba-8b33-41e3-b2a4-620a20afe2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:35.417966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-5aa5d9ba-8b33-41e3-b2a4-620a20afe2c1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:35.420091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5aa5d9ba-8b33-41e3-b2a4-620a20afe2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:35.472599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:35.481360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30e499-9aa4-466f-ad42-755ec11465d2 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:35.493239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 514/1006] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:35.524299 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d271c45-877e-45be-9ec7-754af5f1321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:35.524857 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d271c45-877e-45be-9ec7-754af5f1321c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:35.525148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d271c45-877e-45be-9ec7-754af5f1321c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:35.525404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d271c45-877e-45be-9ec7-754af5f1321c None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:35.526036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d271c45-877e-45be-9ec7-754af5f1321c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:35.526341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d271c45-877e-45be-9ec7-754af5f1321c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:35.527806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d271c45-877e-45be-9ec7-754af5f1321c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:35.537688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 493/1007] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 22 17:06:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:35.538806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5d9ba-8b33-41e3-b2a4-620a20afe2c1 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:35.647286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5d9ba-8b33-41e3-b2a4-620a20afe2c1 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:35.681749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83eb953-bab1-4cbb-a6da-e09efacd04aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:35.682353 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83eb953-bab1-4cbb-a6da-e09efacd04aa None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:35.682655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83eb953-bab1-4cbb-a6da-e09efacd04aa None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:35.682925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83eb953-bab1-4cbb-a6da-e09efacd04aa None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:35.683546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83eb953-bab1-4cbb-a6da-e09efacd04aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:35.683844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83eb953-bab1-4cbb-a6da-e09efacd04aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:35.685191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83eb953-bab1-4cbb-a6da-e09efacd04aa None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:35.691279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 515/1008] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 22 17:06:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:35.696644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d271c45-877e-45be-9ec7-754af5f1321c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:35.734559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:35.767143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83eb953-bab1-4cbb-a6da-e09efacd04aa None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:35.812004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d271c45-877e-45be-9ec7-754af5f1321c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:35.817984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:35.851597 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98119b4b-56ac-42af-941c-7762b1545228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:35.852215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98119b4b-56ac-42af-941c-7762b1545228 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:35.852498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98119b4b-56ac-42af-941c-7762b1545228 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:35.852863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98119b4b-56ac-42af-941c-7762b1545228 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:35.853548 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98119b4b-56ac-42af-941c-7762b1545228 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:35.854958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98119b4b-56ac-42af-941c-7762b1545228 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:35.878745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83eb953-bab1-4cbb-a6da-e09efacd04aa None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:35.913257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e705da3-f618-472a-8465-798eb05d3338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:35.913860 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e705da3-f618-472a-8465-798eb05d3338 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:35.914138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e705da3-f618-472a-8465-798eb05d3338 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:35.914407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e705da3-f618-472a-8465-798eb05d3338 None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:35.915081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e705da3-f618-472a-8465-798eb05d3338 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:35.916452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e705da3-f618-472a-8465-798eb05d3338 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:35.958388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 494/1009] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:06:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:36.062683 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 516/1010] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:06:35 2022] POST /identity/v3/services => generated 242 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:36.698903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699f1a7-22f0-4796-b4ba-c371d66280e1 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:36.699714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699f1a7-22f0-4796-b4ba-c371d66280e1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:36.699714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699f1a7-22f0-4796-b4ba-c371d66280e1 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:36.702241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 495/1011] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:36.708422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b3c5f-f1f3-4df7-bef9-a5c1dd4520d5 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:36.709052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b3c5f-f1f3-4df7-bef9-a5c1dd4520d5 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:36.709052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b3c5f-f1f3-4df7-bef9-a5c1dd4520d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:36.728128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e74b3c5f-f1f3-4df7-bef9-a5c1dd4520d5 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:36.729356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e74b3c5f-f1f3-4df7-bef9-a5c1dd4520d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:36.848286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 517/1012] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:36 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:37.231956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699f1a7-22f0-4796-b4ba-c371d66280e1 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:37.357815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:37.369146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699f1a7-22f0-4796-b4ba-c371d66280e1 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:37.406804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef86b72-6fcd-42f9-882b-51bdf3c09059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:37.407452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef86b72-6fcd-42f9-882b-51bdf3c09059 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:37.407731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef86b72-6fcd-42f9-882b-51bdf3c09059 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:37.408000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef86b72-6fcd-42f9-882b-51bdf3c09059 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:37.408585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef86b72-6fcd-42f9-882b-51bdf3c09059 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:37.409991 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef86b72-6fcd-42f9-882b-51bdf3c09059 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-4ef86b72-6fcd-42f9-882b-51bdf3c09059 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:37.421688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:37.426905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:37.428125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 496/1013] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:37.429495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b3c5f-f1f3-4df7-bef9-a5c1dd4520d5 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:37.468533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:37.554123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b3c5f-f1f3-4df7-bef9-a5c1dd4520d5 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:37.594160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0579-fe9b-4a66-8744-40d1b02151fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:37.594371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0579-fe9b-4a66-8744-40d1b02151fc None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:37.594657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0579-fe9b-4a66-8744-40d1b02151fc None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:37.594824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0579-fe9b-4a66-8744-40d1b02151fc None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:37.595438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf0579-fe9b-4a66-8744-40d1b02151fc None admin] RBAC: Adding query filter params (name=compute) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:37.595733 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf0579-fe9b-4a66-8744-40d1b02151fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:37.597093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf0579-fe9b-4a66-8744-40d1b02151fc None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:37.607089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 518/1014] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:37.608795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef86b72-6fcd-42f9-882b-51bdf3c09059 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:37.757798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef86b72-6fcd-42f9-882b-51bdf3c09059 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:37.796110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76708c9e-00c8-4394-ad89-482e375c0489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:37.796713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76708c9e-00c8-4394-ad89-482e375c0489 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:37.796985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76708c9e-00c8-4394-ad89-482e375c0489 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:37.797248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76708c9e-00c8-4394-ad89-482e375c0489 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:37.797875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76708c9e-00c8-4394-ad89-482e375c0489 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:37.798170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76708c9e-00c8-4394-ad89-482e375c0489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:37.799586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76708c9e-00c8-4394-ad89-482e375c0489 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:37.805049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 497/1015] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 22 17:06:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:37.811347 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0579-fe9b-4a66-8744-40d1b02151fc None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:37.922770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:37.929063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0579-fe9b-4a66-8744-40d1b02151fc None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:37.971596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7b498-25cc-4ca9-82fd-6603ee39aba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:37.972370 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7b498-25cc-4ca9-82fd-6603ee39aba0 None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:37.972551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7b498-25cc-4ca9-82fd-6603ee39aba0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:37.972817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7b498-25cc-4ca9-82fd-6603ee39aba0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:37.973503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7b498-25cc-4ca9-82fd-6603ee39aba0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:37.974883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7b498-25cc-4ca9-82fd-6603ee39aba0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:38.058604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 519/1016] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:06:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:06:38.071691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c29561-2ecf-4791-9339-af371c08c08c None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:38.071973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c29561-2ecf-4791-9339-af371c08c08c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:38.072233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c29561-2ecf-4791-9339-af371c08c08c None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:38.073915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 498/1017] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:38.080662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84b7a5-19fa-48c5-952e-88c80a75a0ae None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:38.081011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84b7a5-19fa-48c5-952e-88c80a75a0ae None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:38.081292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84b7a5-19fa-48c5-952e-88c80a75a0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:38.103025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1a84b7a5-19fa-48c5-952e-88c80a75a0ae None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:38.104517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1a84b7a5-19fa-48c5-952e-88c80a75a0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:38.115450 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdd94e-abff-4496-b741-d9e9110dede4 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:38.115818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdd94e-abff-4496-b741-d9e9110dede4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:38.116018 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdd94e-abff-4496-b741-d9e9110dede4 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:38.117719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 499/1018] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:38.124585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba746a17-1ef1-46b7-874b-6437f1c06da4 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:38.124925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba746a17-1ef1-46b7-874b-6437f1c06da4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:38.125143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba746a17-1ef1-46b7-874b-6437f1c06da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:38.146559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ba746a17-1ef1-46b7-874b-6437f1c06da4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:38.147813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ba746a17-1ef1-46b7-874b-6437f1c06da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:38.166323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:38.205139 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 520/1019] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:38.238649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 500/1020] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:38.475252 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84b7a5-19fa-48c5-952e-88c80a75a0ae None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:38.513707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba746a17-1ef1-46b7-874b-6437f1c06da4 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:38.570050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:38.593791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84b7a5-19fa-48c5-952e-88c80a75a0ae None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:38.637972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241c85b-1f2f-4f29-95fa-3e2fb280bb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:38.638688 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241c85b-1f2f-4f29-95fa-3e2fb280bb78 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:38.638969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241c85b-1f2f-4f29-95fa-3e2fb280bb78 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:38.639240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241c85b-1f2f-4f29-95fa-3e2fb280bb78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:38.640364 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241c85b-1f2f-4f29-95fa-3e2fb280bb78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:38.641810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241c85b-1f2f-4f29-95fa-3e2fb280bb78 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:38.643568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 521/1021] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:38.648992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241c85b-1f2f-4f29-95fa-3e2fb280bb78 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:38.666513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba746a17-1ef1-46b7-874b-6437f1c06da4 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:38.703135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb23a9-c116-4d2a-a5d4-d6c03c9474aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:38.703769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb23a9-c116-4d2a-a5d4-d6c03c9474aa None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:38.704044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb23a9-c116-4d2a-a5d4-d6c03c9474aa None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:38.704313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb23a9-c116-4d2a-a5d4-d6c03c9474aa None admin] PATH_INFO: `/v3/services/image` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:38.704898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb23a9-c116-4d2a-a5d4-d6c03c9474aa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:38.706269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb23a9-c116-4d2a-a5d4-d6c03c9474aa None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-2ccb23a9-c116-4d2a-a5d4-d6c03c9474aa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:38.714981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:38.716153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:38.717305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 501/1022] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Aug 22 17:06:38 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:38.724132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb23a9-c116-4d2a-a5d4-d6c03c9474aa None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:38.741946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:38.758743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241c85b-1f2f-4f29-95fa-3e2fb280bb78 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:38.793564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c815b-dae2-447f-8dab-e612412282ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:38.794165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c815b-dae2-447f-8dab-e612412282ec None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:38.795007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c815b-dae2-447f-8dab-e612412282ec None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:38.795411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c815b-dae2-447f-8dab-e612412282ec None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:38.795988 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c815b-dae2-447f-8dab-e612412282ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:38.797421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c815b-dae2-447f-8dab-e612412282ec None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:38.833258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb23a9-c116-4d2a-a5d4-d6c03c9474aa None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:38.869209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d186fb9-fd7b-405c-9727-e3f231859b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:38.869813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d186fb9-fd7b-405c-9727-e3f231859b3a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:38.870091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d186fb9-fd7b-405c-9727-e3f231859b3a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:38.870363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d186fb9-fd7b-405c-9727-e3f231859b3a None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:38.871025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d186fb9-fd7b-405c-9727-e3f231859b3a None admin] RBAC: Adding query filter params (name=image) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:38.871328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d186fb9-fd7b-405c-9727-e3f231859b3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:38.873284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d186fb9-fd7b-405c-9727-e3f231859b3a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:38.885318 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 502/1023] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:38.885499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:38.887413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d186fb9-fd7b-405c-9727-e3f231859b3a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:38.894011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 522/1024] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:06:38 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:06:38.981589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:39.005808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d186fb9-fd7b-405c-9727-e3f231859b3a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:39.046430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811f3d4-d805-4041-a77a-245d187ec1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:39.046430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f3d4-d805-4041-a77a-245d187ec1ad None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:39.046430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f3d4-d805-4041-a77a-245d187ec1ad None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:39.046430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f3d4-d805-4041-a77a-245d187ec1ad None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:39.046870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811f3d4-d805-4041-a77a-245d187ec1ad None admin] RBAC: Adding query filter params (type=image) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:39.047121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811f3d4-d805-4041-a77a-245d187ec1ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:39.048483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811f3d4-d805-4041-a77a-245d187ec1ad None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:39.053641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 503/1025] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:39.059638 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c815b-dae2-447f-8dab-e612412282ec None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:39.175229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c815b-dae2-447f-8dab-e612412282ec None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:39.211217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25db12-802c-4ddb-aab2-6733d470633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:39.212252 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25db12-802c-4ddb-aab2-6733d470633e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:39.212757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25db12-802c-4ddb-aab2-6733d470633e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:39.213266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25db12-802c-4ddb-aab2-6733d470633e None admin] PATH_INFO: `/v3/endpoints` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:39.214177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25db12-802c-4ddb-aab2-6733d470633e None admin] RBAC: Adding query filter params (interface=public, service_id=a89cffd9d0dd44a3b4745a5e2d20f796, region_id=RegionOne) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:39.214717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25db12-802c-4ddb-aab2-6733d470633e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:39.216831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25db12-802c-4ddb-aab2-6733d470633e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:39.223112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 523/1026] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 22 17:06:39 2022] GET /identity/v3/endpoints?service_id=a89cffd9d0dd44a3b4745a5e2d20f796&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:39.235857 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811f3d4-d805-4041-a77a-245d187ec1ad None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:39.351502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811f3d4-d805-4041-a77a-245d187ec1ad None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:39.386903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2d030-997f-438f-a52f-9bcf11be97a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:39.387627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2d030-997f-438f-a52f-9bcf11be97a9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:39.387911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2d030-997f-438f-a52f-9bcf11be97a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:39.388705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2d030-997f-438f-a52f-9bcf11be97a9 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:39.389312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2d030-997f-438f-a52f-9bcf11be97a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:39.390709 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2d030-997f-438f-a52f-9bcf11be97a9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:39.402519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 504/1027] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 22 17:06:39 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:06:40.193669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce26d1-2c66-481a-a6dd-faa553b0f224 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:40.194643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce26d1-2c66-481a-a6dd-faa553b0f224 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:40.194643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce26d1-2c66-481a-a6dd-faa553b0f224 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:40.195583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 524/1028] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:40.202058 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10824c2c-c125-4170-a972-6ccac8efcc8c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:40.202734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10824c2c-c125-4170-a972-6ccac8efcc8c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:40.202734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10824c2c-c125-4170-a972-6ccac8efcc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:40.215835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:40.226111 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-10824c2c-c125-4170-a972-6ccac8efcc8c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:40.227490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-10824c2c-c125-4170-a972-6ccac8efcc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:40.331661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 505/1029] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:40.634189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce26d1-2c66-481a-a6dd-faa553b0f224 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:40.703221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:40.746334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce26d1-2c66-481a-a6dd-faa553b0f224 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:40.782391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb473b0-86e1-4e6f-9a10-909362af7bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:40.783053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb473b0-86e1-4e6f-9a10-909362af7bb8 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:40.783334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb473b0-86e1-4e6f-9a10-909362af7bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:40.783604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb473b0-86e1-4e6f-9a10-909362af7bb8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:40.795288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb473b0-86e1-4e6f-9a10-909362af7bb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ddb473b0-86e1-4e6f-9a10-909362af7bb8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:40.799991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:40.800982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:40.801897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:40.803471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 525/1030] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 22 17:06:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:40.807029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10824c2c-c125-4170-a972-6ccac8efcc8c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:40.914523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10824c2c-c125-4170-a972-6ccac8efcc8c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:40.952344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb948afa-8767-4a80-b3ff-1b35c8d60700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:40.953012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948afa-8767-4a80-b3ff-1b35c8d60700 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:40.953340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948afa-8767-4a80-b3ff-1b35c8d60700 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:40.953768 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948afa-8767-4a80-b3ff-1b35c8d60700 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:40.954446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948afa-8767-4a80-b3ff-1b35c8d60700 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:40.954778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948afa-8767-4a80-b3ff-1b35c8d60700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:40.956148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948afa-8767-4a80-b3ff-1b35c8d60700 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:40.958559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:40.965665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 506/1031] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 22 17:06:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:40.971540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb473b0-86e1-4e6f-9a10-909362af7bb8 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:41.070584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148bb70-b48d-4244-8637-b51d8cda85d4 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.070911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148bb70-b48d-4244-8637-b51d8cda85d4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.071186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148bb70-b48d-4244-8637-b51d8cda85d4 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.072744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 507/1032] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:41.078345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494f67d-be60-4775-80dc-dff36a48b335 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.078658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494f67d-be60-4775-80dc-dff36a48b335 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.078930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494f67d-be60-4775-80dc-dff36a48b335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.086534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb473b0-86e1-4e6f-9a10-909362af7bb8 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:41.097430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-0494f67d-be60-4775-80dc-dff36a48b335 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:41.098758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0494f67d-be60-4775-80dc-dff36a48b335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:41.113756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:41.128496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cf189-aba8-465c-b79e-4d60c05cebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:41.129281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cf189-aba8-465c-b79e-4d60c05cebf2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.129563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cf189-aba8-465c-b79e-4d60c05cebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.129848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cf189-aba8-465c-b79e-4d60c05cebf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.142288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cf189-aba8-465c-b79e-4d60c05cebf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:41.143852 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cf189-aba8-465c-b79e-4d60c05cebf2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-a07cf189-aba8-465c-b79e-4d60c05cebf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:41.158962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:41.160189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:41.162323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:41.162323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:41.162323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:41.162323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 526/1033] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:41.166048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cf189-aba8-465c-b79e-4d60c05cebf2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:41.183913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 508/1034] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:41.279326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cf189-aba8-465c-b79e-4d60c05cebf2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:41.314925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e62d7-4cfa-4918-aa6f-5cf593d3455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:41.315538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e62d7-4cfa-4918-aa6f-5cf593d3455e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.315811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e62d7-4cfa-4918-aa6f-5cf593d3455e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.316075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e62d7-4cfa-4918-aa6f-5cf593d3455e None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.316707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e62d7-4cfa-4918-aa6f-5cf593d3455e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:41.317001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e62d7-4cfa-4918-aa6f-5cf593d3455e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:41.318463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e62d7-4cfa-4918-aa6f-5cf593d3455e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:41.336311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 527/1035] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:41.338454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494f67d-be60-4775-80dc-dff36a48b335 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:41.443751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494f67d-be60-4775-80dc-dff36a48b335 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:41.470655 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:41.475520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e62d7-4cfa-4918-aa6f-5cf593d3455e None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:41.479566 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18381c30-5ab6-47c0-9e5a-e185204edae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:41.480203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18381c30-5ab6-47c0-9e5a-e185204edae6 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.485177 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18381c30-5ab6-47c0-9e5a-e185204edae6 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.485177 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18381c30-5ab6-47c0-9e5a-e185204edae6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.490959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18381c30-5ab6-47c0-9e5a-e185204edae6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:41.492443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18381c30-5ab6-47c0-9e5a-e185204edae6 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-18381c30-5ab6-47c0-9e5a-e185204edae6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:41.508417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:41.514611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 509/1036] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 22 17:06:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222973cf-2493-4634-ba01-98b2e0f3419d None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.515902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222973cf-2493-4634-ba01-98b2e0f3419d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.516649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222973cf-2493-4634-ba01-98b2e0f3419d None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.516649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 510/1037] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:41.517967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222973cf-2493-4634-ba01-98b2e0f3419d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:41.519326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:41.588990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e62d7-4cfa-4918-aa6f-5cf593d3455e None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:41.625440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16912a2b-0029-49d1-a4ec-1128cb5e99d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:41.625667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222973cf-2493-4634-ba01-98b2e0f3419d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:41.626077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16912a2b-0029-49d1-a4ec-1128cb5e99d7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.626348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16912a2b-0029-49d1-a4ec-1128cb5e99d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.626668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16912a2b-0029-49d1-a4ec-1128cb5e99d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.627958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16912a2b-0029-49d1-a4ec-1128cb5e99d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:41.629396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16912a2b-0029-49d1-a4ec-1128cb5e99d7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:41.631010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 528/1038] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:41.631010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7914e6-1887-40bf-9fd6-aa7ba2e01070 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.631010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7914e6-1887-40bf-9fd6-aa7ba2e01070 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.631010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7914e6-1887-40bf-9fd6-aa7ba2e01070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.651462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-3f7914e6-1887-40bf-9fd6-aa7ba2e01070 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:41.652850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3f7914e6-1887-40bf-9fd6-aa7ba2e01070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:41.663819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4552c-ae06-44d1-afaf-437fdc15cf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:41.664413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4552c-ae06-44d1-afaf-437fdc15cf59 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.664688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4552c-ae06-44d1-afaf-437fdc15cf59 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.664969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4552c-ae06-44d1-afaf-437fdc15cf59 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.665669 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4552c-ae06-44d1-afaf-437fdc15cf59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:41.665970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4552c-ae06-44d1-afaf-437fdc15cf59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:41.667405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4552c-ae06-44d1-afaf-437fdc15cf59 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:41.684792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 511/1039] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 22 17:06:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:41.685897 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4552c-ae06-44d1-afaf-437fdc15cf59 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:41.730036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 529/1040] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:41.731226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7914e6-1887-40bf-9fd6-aa7ba2e01070 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:41.735574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:41.766114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:41.796672 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4552c-ae06-44d1-afaf-437fdc15cf59 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:41.838700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ab6ca-a1ba-48b4-8a24-c50f83d5b8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:41.839317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ab6ca-a1ba-48b4-8a24-c50f83d5b8ec None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.839593 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ab6ca-a1ba-48b4-8a24-c50f83d5b8ec None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.839861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ab6ca-a1ba-48b4-8a24-c50f83d5b8ec None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.840505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ab6ca-a1ba-48b4-8a24-c50f83d5b8ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:41.842533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ab6ca-a1ba-48b4-8a24-c50f83d5b8ec None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:41.861031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7914e6-1887-40bf-9fd6-aa7ba2e01070 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:41.896088 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545789ea-5014-4e27-942e-3ebc432d74fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:41.896719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545789ea-5014-4e27-942e-3ebc432d74fe None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:41.896993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545789ea-5014-4e27-942e-3ebc432d74fe None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:41.897257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545789ea-5014-4e27-942e-3ebc432d74fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:41.970026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545789ea-5014-4e27-942e-3ebc432d74fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:41.971524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545789ea-5014-4e27-942e-3ebc432d74fe None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:42.001374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:42.039188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 512/1041] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:06:41 2022] POST /identity/v3/users => generated 285 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:42.082695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ab6ca-a1ba-48b4-8a24-c50f83d5b8ec None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:42.082865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 530/1042] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:06:42.084749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545789ea-5014-4e27-942e-3ebc432d74fe None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:42.167138 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ab6ca-a1ba-48b4-8a24-c50f83d5b8ec None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:42.169477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:42.217153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b449-2a21-417e-b3f1-d9df18046ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:42.217810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b449-2a21-417e-b3f1-d9df18046ed2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:42.218089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b449-2a21-417e-b3f1-d9df18046ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:42.218359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b449-2a21-417e-b3f1-d9df18046ed2 None admin] PATH_INFO: `/v3/services/image` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:42.219056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8b449-2a21-417e-b3f1-d9df18046ed2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:42.221236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8b449-2a21-417e-b3f1-d9df18046ed2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:42.230971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545789ea-5014-4e27-942e-3ebc432d74fe None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-16e8b449-2a21-417e-b3f1-d9df18046ed2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:42.233584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:42.235777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:42.237642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 513/1043] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Aug 22 17:06:42 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:42.296271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b449-2a21-417e-b3f1-d9df18046ed2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:42.296388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:42.296388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37640cc7-2fef-4e46-b159-4ae15fc01b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:42.296388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37640cc7-2fef-4e46-b159-4ae15fc01b36 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:42.296388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37640cc7-2fef-4e46-b159-4ae15fc01b36 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:42.297013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37640cc7-2fef-4e46-b159-4ae15fc01b36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:42.297013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37640cc7-2fef-4e46-b159-4ae15fc01b36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:42.297013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37640cc7-2fef-4e46-b159-4ae15fc01b36 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-37640cc7-2fef-4e46-b159-4ae15fc01b36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:42.306199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:42.307680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:42.308797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:42.308797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:42.308797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:42.308797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 531/1044] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:42.316532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37640cc7-2fef-4e46-b159-4ae15fc01b36 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:42.369700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b449-2a21-417e-b3f1-d9df18046ed2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:42.411592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f9da1-589a-447c-8bed-81a76410def7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:42.412238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9da1-589a-447c-8bed-81a76410def7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:42.412519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9da1-589a-447c-8bed-81a76410def7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:42.412794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9da1-589a-447c-8bed-81a76410def7 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:42.413434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f9da1-589a-447c-8bed-81a76410def7 None admin] RBAC: Adding query filter params (name=image) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:42.413739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f9da1-589a-447c-8bed-81a76410def7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:42.415333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f9da1-589a-447c-8bed-81a76410def7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:42.421667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 514/1045] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:42.427244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f9da1-589a-447c-8bed-81a76410def7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:42.435529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37640cc7-2fef-4e46-b159-4ae15fc01b36 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:42.460692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:42.473083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db9751-b284-4d74-af20-b8269a2792e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:42.474533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db9751-b284-4d74-af20-b8269a2792e2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:42.474533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db9751-b284-4d74-af20-b8269a2792e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:42.474533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db9751-b284-4d74-af20-b8269a2792e2 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:42.474919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db9751-b284-4d74-af20-b8269a2792e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:42.475218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db9751-b284-4d74-af20-b8269a2792e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:42.476593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db9751-b284-4d74-af20-b8269a2792e2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:42.493505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 532/1046] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:42.496711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db9751-b284-4d74-af20-b8269a2792e2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:42.552056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:42.561675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f9da1-589a-447c-8bed-81a76410def7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:42.606068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433012f-cfd9-44a7-a2ad-b4a6e1f188e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:42.607303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433012f-cfd9-44a7-a2ad-b4a6e1f188e2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:42.607535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433012f-cfd9-44a7-a2ad-b4a6e1f188e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:42.607905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433012f-cfd9-44a7-a2ad-b4a6e1f188e2 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:42.608520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433012f-cfd9-44a7-a2ad-b4a6e1f188e2 None admin] RBAC: Adding query filter params (type=image) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:42.608824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433012f-cfd9-44a7-a2ad-b4a6e1f188e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:42.610243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433012f-cfd9-44a7-a2ad-b4a6e1f188e2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:42.615648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 515/1047] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:42.622213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433012f-cfd9-44a7-a2ad-b4a6e1f188e2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:42.632524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db9751-b284-4d74-af20-b8269a2792e2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:42.677957 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cf732-842c-4147-85dc-d5996d80006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:42.678772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cf732-842c-4147-85dc-d5996d80006d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:42.679045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cf732-842c-4147-85dc-d5996d80006d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:42.679604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cf732-842c-4147-85dc-d5996d80006d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:42.688890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cf732-842c-4147-85dc-d5996d80006d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:42.690359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cf732-842c-4147-85dc-d5996d80006d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ab6cf732-842c-4147-85dc-d5996d80006d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:42.706529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:42.707524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:42.708424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:42.708424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:42.708424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:42.708424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:42.708424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:42.708424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:42.708424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:42.708424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:42.714152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 533/1048] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:42.714703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cf732-842c-4147-85dc-d5996d80006d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:42.737554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:42.745194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433012f-cfd9-44a7-a2ad-b4a6e1f188e2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:42.786864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976e795-71eb-4fe3-bcfb-06ccc0be4b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:42.787662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976e795-71eb-4fe3-bcfb-06ccc0be4b1d None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:42.788039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976e795-71eb-4fe3-bcfb-06ccc0be4b1d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:42.788415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976e795-71eb-4fe3-bcfb-06ccc0be4b1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:42.789201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976e795-71eb-4fe3-bcfb-06ccc0be4b1d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:42.791446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976e795-71eb-4fe3-bcfb-06ccc0be4b1d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:42.834164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cf732-842c-4147-85dc-d5996d80006d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:42.836432 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:42.868500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70c7c5-b84c-4907-acce-bcdbef2b00b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:42.869098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70c7c5-b84c-4907-acce-bcdbef2b00b3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:42.869372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70c7c5-b84c-4907-acce-bcdbef2b00b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:42.869642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70c7c5-b84c-4907-acce-bcdbef2b00b3 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:42.870317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70c7c5-b84c-4907-acce-bcdbef2b00b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:42.870651 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70c7c5-b84c-4907-acce-bcdbef2b00b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:42.872071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70c7c5-b84c-4907-acce-bcdbef2b00b3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:42.884868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 534/1049] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:42.891809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70c7c5-b84c-4907-acce-bcdbef2b00b3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:42.927457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 516/1050] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:06:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:43.031802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70c7c5-b84c-4907-acce-bcdbef2b00b3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:43.079436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6783264-9e6e-44ff-92a2-bc0660cdaa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:43.080045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6783264-9e6e-44ff-92a2-bc0660cdaa0a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:43.080322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6783264-9e6e-44ff-92a2-bc0660cdaa0a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:43.080595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6783264-9e6e-44ff-92a2-bc0660cdaa0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:43.081316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6783264-9e6e-44ff-92a2-bc0660cdaa0a None admin] RBAC: Adding query filter params (role.id=ccb62dfdcc3947e9bdb4bba0f88e4e10, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=9628118c1df64fa4a7ea8ce37549d8b3) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:43.081622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6783264-9e6e-44ff-92a2-bc0660cdaa0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:43.083093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6783264-9e6e-44ff-92a2-bc0660cdaa0a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:43.088924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 535/1051] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:42 2022] GET /identity/v3/role_assignments?user.id=9628118c1df64fa4a7ea8ce37549d8b3&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=ccb62dfdcc3947e9bdb4bba0f88e4e10 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:44.154143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0a4cd-a8ec-47fc-816a-294352f9ffdc None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:44.155006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0a4cd-a8ec-47fc-816a-294352f9ffdc None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:44.155006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0a4cd-a8ec-47fc-816a-294352f9ffdc None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:44.156114 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 517/1052] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:44.161498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9d144-3dca-4cab-94e7-53b7cfbc0753 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:44.161928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9d144-3dca-4cab-94e7-53b7cfbc0753 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:44.162113 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9d144-3dca-4cab-94e7-53b7cfbc0753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:44.181357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-31f9d144-3dca-4cab-94e7-53b7cfbc0753 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:44.182615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-31f9d144-3dca-4cab-94e7-53b7cfbc0753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:44.205266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:44.260296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 536/1053] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:44.536941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0a4cd-a8ec-47fc-816a-294352f9ffdc None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:44.654177 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0a4cd-a8ec-47fc-816a-294352f9ffdc None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:44.691100 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf24ed7-37c9-43fa-b9f3-e2669c2349d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:44.691732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24ed7-37c9-43fa-b9f3-e2669c2349d7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:44.692009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24ed7-37c9-43fa-b9f3-e2669c2349d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:44.692280 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24ed7-37c9-43fa-b9f3-e2669c2349d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:44.703044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf24ed7-37c9-43fa-b9f3-e2669c2349d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-bdf24ed7-37c9-43fa-b9f3-e2669c2349d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:44.707756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:44.708788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:44.711522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 518/1054] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:44.715265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9d144-3dca-4cab-94e7-53b7cfbc0753 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:44.819001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9d144-3dca-4cab-94e7-53b7cfbc0753 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:44.855434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaef184-2a70-4fbd-8e2f-2dd11cc8a520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:44.856375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaef184-2a70-4fbd-8e2f-2dd11cc8a520 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:44.857562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaef184-2a70-4fbd-8e2f-2dd11cc8a520 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:44.858158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaef184-2a70-4fbd-8e2f-2dd11cc8a520 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:44.859061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaef184-2a70-4fbd-8e2f-2dd11cc8a520 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:44.859546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaef184-2a70-4fbd-8e2f-2dd11cc8a520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:44.865351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaef184-2a70-4fbd-8e2f-2dd11cc8a520 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:44.884381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 537/1055] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:06:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:44.891870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf24ed7-37c9-43fa-b9f3-e2669c2349d7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:44.906242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:44.944645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6661cf-f31d-43b9-aea9-2db6ee77031e None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:44.944957 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6661cf-f31d-43b9-aea9-2db6ee77031e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:44.945232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6661cf-f31d-43b9-aea9-2db6ee77031e None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:44.946623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 538/1056] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:44.952130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d9369-e288-4861-8254-b536ea96921e None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:44.952412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d9369-e288-4861-8254-b536ea96921e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:44.952690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d9369-e288-4861-8254-b536ea96921e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:44.971769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-162d9369-e288-4861-8254-b536ea96921e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:44.972972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-162d9369-e288-4861-8254-b536ea96921e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:44.985597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:45.006298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf24ed7-37c9-43fa-b9f3-e2669c2349d7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:45.043853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31370bb6-1779-4468-8306-cf870b881bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:45.045359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31370bb6-1779-4468-8306-cf870b881bb0 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:45.045468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31370bb6-1779-4468-8306-cf870b881bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:45.045756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31370bb6-1779-4468-8306-cf870b881bb0 None admin] PATH_INFO: `/v3/users/0cb9111cfdf140859312ef92be611ed8` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:45.050529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 539/1057] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:45.050529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef9bdb-211a-42c9-9e09-d730291ca8fd None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:45.050931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef9bdb-211a-42c9-9e09-d730291ca8fd None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:45.051047 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef9bdb-211a-42c9-9e09-d730291ca8fd None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:45.052478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 540/1058] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:45.057160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddf61e-34ee-4ded-a132-7d78d8ca00a0 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:45.057447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddf61e-34ee-4ded-a132-7d78d8ca00a0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:45.057763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddf61e-34ee-4ded-a132-7d78d8ca00a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:45.060986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31370bb6-1779-4468-8306-cf870b881bb0 None admin] RBAC: Authorizing `identity:get_user(user_id=0cb9111cfdf140859312ef92be611ed8)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:45.062439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31370bb6-1779-4468-8306-cf870b881bb0 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:45.064551 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 519/1059] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:06:44 2022] GET /identity/v3/users/0cb9111cfdf140859312ef92be611ed8 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:45.070955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31370bb6-1779-4468-8306-cf870b881bb0 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:45.081190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d9ddf61e-34ee-4ded-a132-7d78d8ca00a0 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:45.082381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d9ddf61e-34ee-4ded-a132-7d78d8ca00a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:45.145328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:45.158466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 541/1060] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:45.184081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31370bb6-1779-4468-8306-cf870b881bb0 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:45.221247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b1a81-7023-43ea-92da-848080fe136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:45.222177 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b1a81-7023-43ea-92da-848080fe136c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:45.222685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b1a81-7023-43ea-92da-848080fe136c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:45.223135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b1a81-7023-43ea-92da-848080fe136c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:45.296920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b1a81-7023-43ea-92da-848080fe136c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:45.298738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b1a81-7023-43ea-92da-848080fe136c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:45.322350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddf61e-34ee-4ded-a132-7d78d8ca00a0 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:45.374955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:45.407916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:45.413471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 520/1061] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 17:06:45.419491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b1a81-7023-43ea-92da-848080fe136c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:45.434716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddf61e-34ee-4ded-a132-7d78d8ca00a0 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:45.470680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe3f5e-eb01-4740-b2d0-f4f6ae29e24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:45.471293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe3f5e-eb01-4740-b2d0-f4f6ae29e24b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:45.471574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe3f5e-eb01-4740-b2d0-f4f6ae29e24b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:45.471848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe3f5e-eb01-4740-b2d0-f4f6ae29e24b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:45.521115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b1a81-7023-43ea-92da-848080fe136c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:45.545995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe3f5e-eb01-4740-b2d0-f4f6ae29e24b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:45.547671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe3f5e-eb01-4740-b2d0-f4f6ae29e24b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:45.555533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd042b2-aee2-4966-9b04-51c32c907492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:45.556161 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd042b2-aee2-4966-9b04-51c32c907492 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:45.556434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd042b2-aee2-4966-9b04-51c32c907492 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:45.556700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd042b2-aee2-4966-9b04-51c32c907492 None admin] PATH_INFO: `/v3/projects/af769008beb142f9ac1cb545b08acb1a` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:45.557806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd042b2-aee2-4966-9b04-51c32c907492 None admin] RBAC: Authorizing `identity:get_project(project_id=af769008beb142f9ac1cb545b08acb1a)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:45.559239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd042b2-aee2-4966-9b04-51c32c907492 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:45.561689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 521/1062] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:06:45 2022] GET /identity/v3/projects/af769008beb142f9ac1cb545b08acb1a => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:45.562454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd042b2-aee2-4966-9b04-51c32c907492 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:45.601990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:45.637541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:45.663045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 542/1063] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 17:06:45.663890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe3f5e-eb01-4740-b2d0-f4f6ae29e24b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:45.673893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd042b2-aee2-4966-9b04-51c32c907492 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:45.708748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97085db6-b967-4ea3-bc5f-7ff09810adee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:45.709384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97085db6-b967-4ea3-bc5f-7ff09810adee None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:45.709651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97085db6-b967-4ea3-bc5f-7ff09810adee None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:45.709921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97085db6-b967-4ea3-bc5f-7ff09810adee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:45.720764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97085db6-b967-4ea3-bc5f-7ff09810adee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-97085db6-b967-4ea3-bc5f-7ff09810adee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:45.725460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:45.726692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 522/1064] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 22 17:06:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:45.729147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97085db6-b967-4ea3-bc5f-7ff09810adee None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:45.775288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe3f5e-eb01-4740-b2d0-f4f6ae29e24b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:45.817785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa195929-5de7-4ae6-a64e-64007ebbccea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:45.818401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa195929-5de7-4ae6-a64e-64007ebbccea None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:45.818707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa195929-5de7-4ae6-a64e-64007ebbccea None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:45.818982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa195929-5de7-4ae6-a64e-64007ebbccea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:45.819710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa195929-5de7-4ae6-a64e-64007ebbccea None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=af769008beb142f9ac1cb545b08acb1a, user.id=0cb9111cfdf140859312ef92be611ed8) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:45.820029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa195929-5de7-4ae6-a64e-64007ebbccea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:45.821481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa195929-5de7-4ae6-a64e-64007ebbccea None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:45.827322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 543/1065] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:45 2022] GET /identity/v3/role_assignments?user.id=0cb9111cfdf140859312ef92be611ed8&scope.project.id=af769008beb142f9ac1cb545b08acb1a&role.id=4b10782d99b74f5bb453b82286012e0c => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:45.828135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa195929-5de7-4ae6-a64e-64007ebbccea None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:45.839894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:45.850950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97085db6-b967-4ea3-bc5f-7ff09810adee None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:45.879634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:45.892602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b04da-0c30-4842-95c3-603347956b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:45.895112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b04da-0c30-4842-95c3-603347956b87 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:45.895112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b04da-0c30-4842-95c3-603347956b87 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:45.895112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b04da-0c30-4842-95c3-603347956b87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:45.906370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b04da-0c30-4842-95c3-603347956b87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:45.907964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b04da-0c30-4842-95c3-603347956b87 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-a95b04da-0c30-4842-95c3-603347956b87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:45.925852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:45.927131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:45.928037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:45.928037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:45.928037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:45.928037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 523/1066] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:45.934952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b04da-0c30-4842-95c3-603347956b87 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:45.950928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa195929-5de7-4ae6-a64e-64007ebbccea None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:45.996557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169245d7-d9a8-4a06-b670-006a9f6b8e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:45.997393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169245d7-d9a8-4a06-b670-006a9f6b8e9b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:45.997687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169245d7-d9a8-4a06-b670-006a9f6b8e9b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:45.997968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169245d7-d9a8-4a06-b670-006a9f6b8e9b None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:45.998696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169245d7-d9a8-4a06-b670-006a9f6b8e9b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:45.999033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169245d7-d9a8-4a06-b670-006a9f6b8e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:46.000427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169245d7-d9a8-4a06-b670-006a9f6b8e9b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:46.010538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 544/1067] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 22 17:06:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:46.018934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169245d7-d9a8-4a06-b670-006a9f6b8e9b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:46.065605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b04da-0c30-4842-95c3-603347956b87 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:46.105713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0872c-5bc3-4017-a015-95cd25f355d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:46.106368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0872c-5bc3-4017-a015-95cd25f355d5 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:46.107098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0872c-5bc3-4017-a015-95cd25f355d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:46.107383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0872c-5bc3-4017-a015-95cd25f355d5 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:46.108038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0872c-5bc3-4017-a015-95cd25f355d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:46.108622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0872c-5bc3-4017-a015-95cd25f355d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:46.111002 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0872c-5bc3-4017-a015-95cd25f355d5 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:46.134637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 524/1068] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:46.152643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169245d7-d9a8-4a06-b670-006a9f6b8e9b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:46.155624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:46.193753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb25be2-7160-445b-9fb5-124000231edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:46.194387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb25be2-7160-445b-9fb5-124000231edd None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:46.194698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb25be2-7160-445b-9fb5-124000231edd None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:46.194972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb25be2-7160-445b-9fb5-124000231edd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:46.208386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb25be2-7160-445b-9fb5-124000231edd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:46.209880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb25be2-7160-445b-9fb5-124000231edd None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-fbb25be2-7160-445b-9fb5-124000231edd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:46.227046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:46.228170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:46.229405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:46.229405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:46.229405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:46.229405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 545/1069] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:46.234737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0872c-5bc3-4017-a015-95cd25f355d5 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:46.254153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:46.402261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0872c-5bc3-4017-a015-95cd25f355d5 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:46.447555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0258541-4337-4b30-bf3d-aac9e8a5f744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:46.448176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0258541-4337-4b30-bf3d-aac9e8a5f744 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:46.448455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0258541-4337-4b30-bf3d-aac9e8a5f744 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:46.448730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0258541-4337-4b30-bf3d-aac9e8a5f744 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:46.449387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0258541-4337-4b30-bf3d-aac9e8a5f744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:46.449694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0258541-4337-4b30-bf3d-aac9e8a5f744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:46.451678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0258541-4337-4b30-bf3d-aac9e8a5f744 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:46.464370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 525/1070] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:46.470297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb25be2-7160-445b-9fb5-124000231edd None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:46.572818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb25be2-7160-445b-9fb5-124000231edd None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:46.608147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5bcb6-aab8-4edf-ba13-a42fa592a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:46.608788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5bcb6-aab8-4edf-ba13-a42fa592a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:46.609064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5bcb6-aab8-4edf-ba13-a42fa592a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:46.609336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5bcb6-aab8-4edf-ba13-a42fa592a7c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:46.618709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5bcb6-aab8-4edf-ba13-a42fa592a7c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:46.620264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5bcb6-aab8-4edf-ba13-a42fa592a7c3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-58e5bcb6-aab8-4edf-ba13-a42fa592a7c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:46.633910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:46.635106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:46.636411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 546/1071] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 22 17:06:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:46.642020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0258541-4337-4b30-bf3d-aac9e8a5f744 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:46.713005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:46.746404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0258541-4337-4b30-bf3d-aac9e8a5f744 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:46.782005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5c2a7-f4eb-49cf-a352-76e0fbfc6d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:46.782656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5c2a7-f4eb-49cf-a352-76e0fbfc6d7a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:46.782933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5c2a7-f4eb-49cf-a352-76e0fbfc6d7a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:46.783203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5c2a7-f4eb-49cf-a352-76e0fbfc6d7a None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:46.783927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5c2a7-f4eb-49cf-a352-76e0fbfc6d7a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:46.784229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5c2a7-f4eb-49cf-a352-76e0fbfc6d7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:46.785687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5c2a7-f4eb-49cf-a352-76e0fbfc6d7a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:46.805227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 526/1072] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 22 17:06:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:46.812009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5bcb6-aab8-4edf-ba13-a42fa592a7c3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:46.829322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:46.916723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5bcb6-aab8-4edf-ba13-a42fa592a7c3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:46.952480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbf77e-df8d-461d-a206-e9be1723d7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:46.953107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbf77e-df8d-461d-a206-e9be1723d7c2 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:46.953381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbf77e-df8d-461d-a206-e9be1723d7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:46.953647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbf77e-df8d-461d-a206-e9be1723d7c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:46.964452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbf77e-df8d-461d-a206-e9be1723d7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:46.965954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbf77e-df8d-461d-a206-e9be1723d7c2 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-83fbf77e-df8d-461d-a206-e9be1723d7c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:46.980468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:46.982229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:46.983226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:46.983226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:46.983226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:46.983226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 547/1073] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:46.988369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5c2a7-f4eb-49cf-a352-76e0fbfc6d7a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:47.089648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5c2a7-f4eb-49cf-a352-76e0fbfc6d7a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:47.116314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:47.125969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059ae2-e8b6-41fc-8367-0d4a36d9c786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:47.128327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ae2-e8b6-41fc-8367-0d4a36d9c786 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:47.128327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ae2-e8b6-41fc-8367-0d4a36d9c786 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:47.128327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ae2-e8b6-41fc-8367-0d4a36d9c786 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:47.128327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ae2-e8b6-41fc-8367-0d4a36d9c786 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:47.128327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ae2-e8b6-41fc-8367-0d4a36d9c786 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:47.129460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ae2-e8b6-41fc-8367-0d4a36d9c786 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:47.142075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 527/1074] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:47.147731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbf77e-df8d-461d-a206-e9be1723d7c2 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:47.215433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:47.251992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbf77e-df8d-461d-a206-e9be1723d7c2 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:47.287880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ac94c-5a16-4832-978d-e8e0b5f98bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:47.288512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ac94c-5a16-4832-978d-e8e0b5f98bcc None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:47.288787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ac94c-5a16-4832-978d-e8e0b5f98bcc None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:47.289078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ac94c-5a16-4832-978d-e8e0b5f98bcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:47.297317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ac94c-5a16-4832-978d-e8e0b5f98bcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:47.298795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ac94c-5a16-4832-978d-e8e0b5f98bcc None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-8e1ac94c-5a16-4832-978d-e8e0b5f98bcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:47.309966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:47.310961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:47.312209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:47.312209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:47.312209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:47.312209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:47.312209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:47.312209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:47.312209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:47.312209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:47.312209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 548/1075] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:47.316925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059ae2-e8b6-41fc-8367-0d4a36d9c786 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:47.429890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059ae2-e8b6-41fc-8367-0d4a36d9c786 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:47.465553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a35127-9810-44dc-94b5-878b13b77db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:47.466625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a35127-9810-44dc-94b5-878b13b77db2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:47.467006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a35127-9810-44dc-94b5-878b13b77db2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:47.467266 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a35127-9810-44dc-94b5-878b13b77db2 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:47.467963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a35127-9810-44dc-94b5-878b13b77db2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:47.468291 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a35127-9810-44dc-94b5-878b13b77db2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:47.469733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a35127-9810-44dc-94b5-878b13b77db2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:47.487844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 528/1076] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:47.489107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ac94c-5a16-4832-978d-e8e0b5f98bcc None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:47.594427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ac94c-5a16-4832-978d-e8e0b5f98bcc None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:47.621033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:47.631675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498aa0b-1b94-45a4-bf57-7fef6e9ef3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:47.632333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498aa0b-1b94-45a4-bf57-7fef6e9ef3a8 None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:47.635639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498aa0b-1b94-45a4-bf57-7fef6e9ef3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:47.635639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498aa0b-1b94-45a4-bf57-7fef6e9ef3a8 None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/9628118c1df64fa4a7ea8ce37549d8b3/roles/ccb62dfdcc3947e9bdb4bba0f88e4e10` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:47.635639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498aa0b-1b94-45a4-bf57-7fef6e9ef3a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=9628118c1df64fa4a7ea8ce37549d8b3, role_id=ccb62dfdcc3947e9bdb4bba0f88e4e10)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:47.636423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498aa0b-1b94-45a4-bf57-7fef6e9ef3a8 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:47.723520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 549/1077] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:06:47 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/9628118c1df64fa4a7ea8ce37549d8b3/roles/ccb62dfdcc3947e9bdb4bba0f88e4e10 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:06:47.750421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490370ec-fadb-46a3-a10a-83ed00ee22ed None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:47.750769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490370ec-fadb-46a3-a10a-83ed00ee22ed None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:47.751068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490370ec-fadb-46a3-a10a-83ed00ee22ed None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:47.752467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 529/1078] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:47.758693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1331d-3108-49b7-bcb1-eb9f6dcae341 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:47.758976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1331d-3108-49b7-bcb1-eb9f6dcae341 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:47.759275 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1331d-3108-49b7-bcb1-eb9f6dcae341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:47.780770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-65a1331d-3108-49b7-bcb1-eb9f6dcae341 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:47.782004 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-65a1331d-3108-49b7-bcb1-eb9f6dcae341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:47.860512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 550/1079] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:48.131281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490370ec-fadb-46a3-a10a-83ed00ee22ed None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:48.181917 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:48.239982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490370ec-fadb-46a3-a10a-83ed00ee22ed None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:48.279545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dd8cc-256e-47f9-a57c-483e7d1a68e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:48.280176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dd8cc-256e-47f9-a57c-483e7d1a68e3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:48.280452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dd8cc-256e-47f9-a57c-483e7d1a68e3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:48.280725 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dd8cc-256e-47f9-a57c-483e7d1a68e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:48.291186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dd8cc-256e-47f9-a57c-483e7d1a68e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-fe4dd8cc-256e-47f9-a57c-483e7d1a68e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:48.296580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:48.297722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:48.298766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 530/1080] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:48.303754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1331d-3108-49b7-bcb1-eb9f6dcae341 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:48.418561 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1331d-3108-49b7-bcb1-eb9f6dcae341 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:48.436966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:48.457392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227888c-92c8-4048-816a-ae44e83cee84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:48.457392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227888c-92c8-4048-816a-ae44e83cee84 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:48.457392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227888c-92c8-4048-816a-ae44e83cee84 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:48.457392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227888c-92c8-4048-816a-ae44e83cee84 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:48.457780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227888c-92c8-4048-816a-ae44e83cee84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:48.457780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227888c-92c8-4048-816a-ae44e83cee84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:48.457780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227888c-92c8-4048-816a-ae44e83cee84 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:48.470855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 551/1081] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:06:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:48.472526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dd8cc-256e-47f9-a57c-483e7d1a68e3 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:48.588304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dd8cc-256e-47f9-a57c-483e7d1a68e3 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:48.598838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:48.628050 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c90337-6924-4202-99b0-072f094497ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:48.628685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c90337-6924-4202-99b0-072f094497ae None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:48.629021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c90337-6924-4202-99b0-072f094497ae None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:48.629241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c90337-6924-4202-99b0-072f094497ae None admin] PATH_INFO: `/v3/users/0cb9111cfdf140859312ef92be611ed8` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:48.630529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c90337-6924-4202-99b0-072f094497ae None admin] RBAC: Authorizing `identity:get_user(user_id=0cb9111cfdf140859312ef92be611ed8)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:48.631967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c90337-6924-4202-99b0-072f094497ae None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:48.633783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 531/1082] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:06:48 2022] GET /identity/v3/users/0cb9111cfdf140859312ef92be611ed8 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:48.639760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227888c-92c8-4048-816a-ae44e83cee84 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:48.754045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227888c-92c8-4048-816a-ae44e83cee84 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:48.788107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4ecbc-4856-4a33-af0b-6cb2f89961f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:48.788724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4ecbc-4856-4a33-af0b-6cb2f89961f6 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:48.788997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4ecbc-4856-4a33-af0b-6cb2f89961f6 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:48.789265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4ecbc-4856-4a33-af0b-6cb2f89961f6 None admin] PATH_INFO: `/v3/projects/af769008beb142f9ac1cb545b08acb1a` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:48.790372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4ecbc-4856-4a33-af0b-6cb2f89961f6 None admin] RBAC: Authorizing `identity:get_project(project_id=af769008beb142f9ac1cb545b08acb1a)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:48.791834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4ecbc-4856-4a33-af0b-6cb2f89961f6 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:48.798515 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 552/1083] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:06:48 2022] GET /identity/v3/projects/af769008beb142f9ac1cb545b08acb1a => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:48.800861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c90337-6924-4202-99b0-072f094497ae None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:48.901066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c90337-6924-4202-99b0-072f094497ae None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:48.935570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abcb66-cdf5-4981-b39f-6f8582db42b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:48.936220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abcb66-cdf5-4981-b39f-6f8582db42b4 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:48.936498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abcb66-cdf5-4981-b39f-6f8582db42b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:48.936769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abcb66-cdf5-4981-b39f-6f8582db42b4 None admin] PATH_INFO: `/v3/projects/af769008beb142f9ac1cb545b08acb1a/users/0cb9111cfdf140859312ef92be611ed8/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:48.938686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abcb66-cdf5-4981-b39f-6f8582db42b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=af769008beb142f9ac1cb545b08acb1a, user_id=0cb9111cfdf140859312ef92be611ed8, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:48.940123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abcb66-cdf5-4981-b39f-6f8582db42b4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:49.033918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 532/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:06:48 2022] PUT /identity/v3/projects/af769008beb142f9ac1cb545b08acb1a/users/0cb9111cfdf140859312ef92be611ed8/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:06:49.654799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc52056-71a5-4bce-b37c-6e251d427fb8 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:49.655319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc52056-71a5-4bce-b37c-6e251d427fb8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:49.655319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc52056-71a5-4bce-b37c-6e251d427fb8 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:49.656669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 553/1085] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:49.661968 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283d903-9a84-48d3-9284-10ccc606a690 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:49.662380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283d903-9a84-48d3-9284-10ccc606a690 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:49.662535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283d903-9a84-48d3-9284-10ccc606a690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:49.681303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2283d903-9a84-48d3-9284-10ccc606a690 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:49.682468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2283d903-9a84-48d3-9284-10ccc606a690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:49.708734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:49.750181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 533/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:50.020021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc52056-71a5-4bce-b37c-6e251d427fb8 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:50.068772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:50.126953 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc52056-71a5-4bce-b37c-6e251d427fb8 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:50.160779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67c8d1-b9c9-4331-9776-c87a21ce729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:50.161398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67c8d1-b9c9-4331-9776-c87a21ce729c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:50.161676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67c8d1-b9c9-4331-9776-c87a21ce729c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:50.161947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67c8d1-b9c9-4331-9776-c87a21ce729c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:50.172752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67c8d1-b9c9-4331-9776-c87a21ce729c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-0c67c8d1-b9c9-4331-9776-c87a21ce729c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:50.177390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:50.178316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:50.179817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 554/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 22 17:06:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:50.185031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283d903-9a84-48d3-9284-10ccc606a690 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:50.284373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283d903-9a84-48d3-9284-10ccc606a690 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:50.318819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c826e-60cb-4df1-a834-fedf16afd360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:50.319424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c826e-60cb-4df1-a834-fedf16afd360 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:50.319697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c826e-60cb-4df1-a834-fedf16afd360 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:50.319967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c826e-60cb-4df1-a834-fedf16afd360 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:50.320619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c826e-60cb-4df1-a834-fedf16afd360 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:50.321514 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c826e-60cb-4df1-a834-fedf16afd360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:50.322293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c826e-60cb-4df1-a834-fedf16afd360 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:50.337391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 534/1088] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 22 17:06:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:50.338636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67c8d1-b9c9-4331-9776-c87a21ce729c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:50.441359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67c8d1-b9c9-4331-9776-c87a21ce729c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:50.448496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:50.477096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e4932-4fac-4aef-bad8-84ead0f7148a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:50.477765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e4932-4fac-4aef-bad8-84ead0f7148a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:50.478013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e4932-4fac-4aef-bad8-84ead0f7148a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:50.478367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e4932-4fac-4aef-bad8-84ead0f7148a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:50.491293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e4932-4fac-4aef-bad8-84ead0f7148a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:50.493068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e4932-4fac-4aef-bad8-84ead0f7148a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-362e4932-4fac-4aef-bad8-84ead0f7148a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:50.509865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:50.511118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:50.513074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:50.513074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:50.513074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:50.513074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 555/1089] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:50.517796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c826e-60cb-4df1-a834-fedf16afd360 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:50.547297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:50.624220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c826e-60cb-4df1-a834-fedf16afd360 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:50.666594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbd184-e74c-42a4-a06e-75fb198b8a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:50.667244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbd184-e74c-42a4-a06e-75fb198b8a6b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:50.667723 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbd184-e74c-42a4-a06e-75fb198b8a6b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:50.667990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbd184-e74c-42a4-a06e-75fb198b8a6b None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:50.668659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbd184-e74c-42a4-a06e-75fb198b8a6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:50.669154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbd184-e74c-42a4-a06e-75fb198b8a6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:50.670635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbd184-e74c-42a4-a06e-75fb198b8a6b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:50.686312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 535/1090] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:50.692373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e4932-4fac-4aef-bad8-84ead0f7148a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:50.819987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e4932-4fac-4aef-bad8-84ead0f7148a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:50.862697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea3b6d-9d49-4e6e-9b03-20d1130f2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:50.863332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea3b6d-9d49-4e6e-9b03-20d1130f2fbe None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:50.863611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea3b6d-9d49-4e6e-9b03-20d1130f2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:50.863890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea3b6d-9d49-4e6e-9b03-20d1130f2fbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:50.872928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea3b6d-9d49-4e6e-9b03-20d1130f2fbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:50.874382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea3b6d-9d49-4e6e-9b03-20d1130f2fbe None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ccea3b6d-9d49-4e6e-9b03-20d1130f2fbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:06:50.889082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:50.890136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:50.891220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:50.891220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:50.891220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:06:50.891220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:06:50.891220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:06:50.891220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:06:50.891220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:06:50.891220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:06:50.891220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 17:06:50.891220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:50.891775 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 556/1091] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 22 17:06:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:06:50.897146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbd184-e74c-42a4-a06e-75fb198b8a6b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:50.957506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bd175-df09-41af-a2d9-743d03a214ef None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:50.957820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bd175-df09-41af-a2d9-743d03a214ef None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:50.958031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bd175-df09-41af-a2d9-743d03a214ef None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:50.959431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 557/1092] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:06:50.964343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1b117-775d-48da-a588-ff34f576152a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:50.964680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1b117-775d-48da-a588-ff34f576152a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:50.964908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1b117-775d-48da-a588-ff34f576152a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:50.985931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-63c1b117-775d-48da-a588-ff34f576152a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:50.987174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-63c1b117-775d-48da-a588-ff34f576152a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:51.002586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:51.010514 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:51.026846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbd184-e74c-42a4-a06e-75fb198b8a6b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:51.062517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 558/1093] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:06:51.066976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dbc54-5013-4398-9173-20819c6a872e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:51.067576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dbc54-5013-4398-9173-20819c6a872e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:51.067849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dbc54-5013-4398-9173-20819c6a872e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:51.068113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dbc54-5013-4398-9173-20819c6a872e None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:51.068835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2dbc54-5013-4398-9173-20819c6a872e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:51.069176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2dbc54-5013-4398-9173-20819c6a872e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:51.070661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2dbc54-5013-4398-9173-20819c6a872e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:51.095152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 536/1094] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 22 17:06:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:51.101078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1b117-775d-48da-a588-ff34f576152a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:51.217494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1b117-775d-48da-a588-ff34f576152a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:51.257063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a57acf-c809-4cc2-a352-9fc5f4e868e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:51.257674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a57acf-c809-4cc2-a352-9fc5f4e868e5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:51.257951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a57acf-c809-4cc2-a352-9fc5f4e868e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:51.258234 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a57acf-c809-4cc2-a352-9fc5f4e868e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:51.277739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:51.342748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a57acf-c809-4cc2-a352-9fc5f4e868e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:51.343759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dbc54-5013-4398-9173-20819c6a872e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:51.344172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a57acf-c809-4cc2-a352-9fc5f4e868e5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:51.455874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dbc54-5013-4398-9173-20819c6a872e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:51.465642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 559/1095] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 17:06:51.472740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a57acf-c809-4cc2-a352-9fc5f4e868e5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:51.500051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe235b-990c-44ca-a1fe-d6102571c46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:51.500693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe235b-990c-44ca-a1fe-d6102571c46f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:51.500974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe235b-990c-44ca-a1fe-d6102571c46f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:51.501284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe235b-990c-44ca-a1fe-d6102571c46f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:51.511842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe235b-990c-44ca-a1fe-d6102571c46f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-32fe235b-990c-44ca-a1fe-d6102571c46f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:06:51.516874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:51.517842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:06:51.518724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 537/1096] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:06:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:51.523907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe235b-990c-44ca-a1fe-d6102571c46f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:51.525550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:51.529472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:51.581671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a57acf-c809-4cc2-a352-9fc5f4e868e5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:51.618186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729b512-1e63-457f-8739-93e5fa404d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:51.618769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729b512-1e63-457f-8739-93e5fa404d9b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:51.619050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729b512-1e63-457f-8739-93e5fa404d9b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:51.619324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729b512-1e63-457f-8739-93e5fa404d9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:51.631194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729b512-1e63-457f-8739-93e5fa404d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:51.632645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729b512-1e63-457f-8739-93e5fa404d9b None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:51.636397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe235b-990c-44ca-a1fe-d6102571c46f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-9729b512-1e63-457f-8739-93e5fa404d9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:06:51.647484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:51.648475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:06:51.649390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:06:51.649390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:06:51.649390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:51.649390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 560/1097] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:51.654728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729b512-1e63-457f-8739-93e5fa404d9b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:51.672316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07a9d3-f1ef-4e9f-aa31-5cc4a258ee06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:51.672920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07a9d3-f1ef-4e9f-aa31-5cc4a258ee06 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:51.673229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07a9d3-f1ef-4e9f-aa31-5cc4a258ee06 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:51.673499 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07a9d3-f1ef-4e9f-aa31-5cc4a258ee06 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:51.674159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07a9d3-f1ef-4e9f-aa31-5cc4a258ee06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:51.674459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07a9d3-f1ef-4e9f-aa31-5cc4a258ee06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:51.675918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07a9d3-f1ef-4e9f-aa31-5cc4a258ee06 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:51.685939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 538/1098] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:06:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:51.691864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07a9d3-f1ef-4e9f-aa31-5cc4a258ee06 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:51.756644 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:51.758347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:51.760441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729b512-1e63-457f-8739-93e5fa404d9b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:51.794895 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07a9d3-f1ef-4e9f-aa31-5cc4a258ee06 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:51.795794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c103d3-08e0-464e-ace3-edae92e3e521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:51.796405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c103d3-08e0-464e-ace3-edae92e3e521 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:51.796682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c103d3-08e0-464e-ace3-edae92e3e521 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:51.796966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c103d3-08e0-464e-ace3-edae92e3e521 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:51.797593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c103d3-08e0-464e-ace3-edae92e3e521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:51.797892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c103d3-08e0-464e-ace3-edae92e3e521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:51.799340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c103d3-08e0-464e-ace3-edae92e3e521 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:51.813172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 561/1099] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:51.820219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c103d3-08e0-464e-ace3-edae92e3e521 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:51.842914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3744c49-4968-43a6-b74f-3ca56be9343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:51.844172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3744c49-4968-43a6-b74f-3ca56be9343e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:51.844454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3744c49-4968-43a6-b74f-3ca56be9343e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:51.844728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3744c49-4968-43a6-b74f-3ca56be9343e None admin] PATH_INFO: `/v3/users/0cb9111cfdf140859312ef92be611ed8` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:51.845921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3744c49-4968-43a6-b74f-3ca56be9343e None admin] RBAC: Authorizing `identity:get_user(user_id=0cb9111cfdf140859312ef92be611ed8)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:51.847400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3744c49-4968-43a6-b74f-3ca56be9343e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:51.849281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 539/1100] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:06:51 2022] GET /identity/v3/users/0cb9111cfdf140859312ef92be611ed8 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:51.856693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3744c49-4968-43a6-b74f-3ca56be9343e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:51.927175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c103d3-08e0-464e-ace3-edae92e3e521 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:51.961231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3744c49-4968-43a6-b74f-3ca56be9343e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:51.962963 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971de3ae-8d30-4d40-a6f0-e3e28c37aa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:51.963597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971de3ae-8d30-4d40-a6f0-e3e28c37aa61 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:51.963870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971de3ae-8d30-4d40-a6f0-e3e28c37aa61 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:51.964140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971de3ae-8d30-4d40-a6f0-e3e28c37aa61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:51.971565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971de3ae-8d30-4d40-a6f0-e3e28c37aa61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:51.973189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971de3ae-8d30-4d40-a6f0-e3e28c37aa61 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-971de3ae-8d30-4d40-a6f0-e3e28c37aa61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:06:51.984976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:06:51.985999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:06:51.986962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:06:51.986962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:06:51.986962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:06:51.986962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:06:51.986962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:06:51.986962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:06:51.986962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:06:51.986962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:06:51.986962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 562/1101] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:06:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:06:51.991854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971de3ae-8d30-4d40-a6f0-e3e28c37aa61 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:51.996799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0171840-8e2f-4fea-ac9b-7596755ad725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:51.997434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0171840-8e2f-4fea-ac9b-7596755ad725 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:51.997714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0171840-8e2f-4fea-ac9b-7596755ad725 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:51.997986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0171840-8e2f-4fea-ac9b-7596755ad725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:52.000708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:52.013087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:52.066766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0171840-8e2f-4fea-ac9b-7596755ad725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:52.068229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0171840-8e2f-4fea-ac9b-7596755ad725 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:52.092822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971de3ae-8d30-4d40-a6f0-e3e28c37aa61 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:52.126836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239af1b-9aea-4ea8-a6d3-ec00886168f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:52.127471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239af1b-9aea-4ea8-a6d3-ec00886168f0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:52.127744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239af1b-9aea-4ea8-a6d3-ec00886168f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:52.128013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239af1b-9aea-4ea8-a6d3-ec00886168f0 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:52.128694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239af1b-9aea-4ea8-a6d3-ec00886168f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:52.128997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239af1b-9aea-4ea8-a6d3-ec00886168f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:52.130425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239af1b-9aea-4ea8-a6d3-ec00886168f0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:52.142363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 563/1102] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:52.148450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239af1b-9aea-4ea8-a6d3-ec00886168f0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:52.170508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 540/1103] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 17:06:52.176531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0171840-8e2f-4fea-ac9b-7596755ad725 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:52.209552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:52.230442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:52.249928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239af1b-9aea-4ea8-a6d3-ec00886168f0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:52.277925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0171840-8e2f-4fea-ac9b-7596755ad725 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:52.286064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b302d6-04f9-44e1-bb00-bb5f1eb8e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:52.286694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b302d6-04f9-44e1-bb00-bb5f1eb8e8e4 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:52.286971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b302d6-04f9-44e1-bb00-bb5f1eb8e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:52.287235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b302d6-04f9-44e1-bb00-bb5f1eb8e8e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:52.287948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b302d6-04f9-44e1-bb00-bb5f1eb8e8e4 None admin] RBAC: Adding query filter params (role.id=ccb62dfdcc3947e9bdb4bba0f88e4e10, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=9628118c1df64fa4a7ea8ce37549d8b3) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:52.288251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b302d6-04f9-44e1-bb00-bb5f1eb8e8e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:52.289678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b302d6-04f9-44e1-bb00-bb5f1eb8e8e4 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:52.296202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 564/1104] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:52 2022] GET /identity/v3/role_assignments?user.id=9628118c1df64fa4a7ea8ce37549d8b3&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=ccb62dfdcc3947e9bdb4bba0f88e4e10 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:52.317852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad68871-aa5c-46ec-9e00-3fe1343c8796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:52.318554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad68871-aa5c-46ec-9e00-3fe1343c8796 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:52.318837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad68871-aa5c-46ec-9e00-3fe1343c8796 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:52.319134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad68871-aa5c-46ec-9e00-3fe1343c8796 None admin] PATH_INFO: `/v3/projects/af769008beb142f9ac1cb545b08acb1a` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:52.320406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad68871-aa5c-46ec-9e00-3fe1343c8796 None admin] RBAC: Authorizing `identity:get_project(project_id=af769008beb142f9ac1cb545b08acb1a)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:52.321913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad68871-aa5c-46ec-9e00-3fe1343c8796 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:52.323888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 541/1105] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:06:52 2022] GET /identity/v3/projects/af769008beb142f9ac1cb545b08acb1a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:06:52.331106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b302d6-04f9-44e1-bb00-bb5f1eb8e8e4 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:52.467064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b302d6-04f9-44e1-bb00-bb5f1eb8e8e4 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:52.502998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1dc5c-b129-46c7-84ad-263176a5dd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:52.503604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1dc5c-b129-46c7-84ad-263176a5dd05 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:52.503879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1dc5c-b129-46c7-84ad-263176a5dd05 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:52.504148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1dc5c-b129-46c7-84ad-263176a5dd05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:52.504873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1dc5c-b129-46c7-84ad-263176a5dd05 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=af769008beb142f9ac1cb545b08acb1a, user.id=0cb9111cfdf140859312ef92be611ed8) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:52.505180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1dc5c-b129-46c7-84ad-263176a5dd05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:52.506645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1dc5c-b129-46c7-84ad-263176a5dd05 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:52.513591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 565/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:52 2022] GET /identity/v3/role_assignments?user.id=0cb9111cfdf140859312ef92be611ed8&scope.project.id=af769008beb142f9ac1cb545b08acb1a&role.id=4b10782d99b74f5bb453b82286012e0c => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:54.476927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d3b3b-025e-4f24-ad78-2fcc2f92a34b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:54.477448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d3b3b-025e-4f24-ad78-2fcc2f92a34b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:54.477538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d3b3b-025e-4f24-ad78-2fcc2f92a34b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:54.478895 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 542/1107] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:54.484821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc95fad-3382-4737-8b2e-7363f402ce58 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:54.485278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc95fad-3382-4737-8b2e-7363f402ce58 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:54.485372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc95fad-3382-4737-8b2e-7363f402ce58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:54.504115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-3cc95fad-3382-4737-8b2e-7363f402ce58 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:54.505296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3cc95fad-3382-4737-8b2e-7363f402ce58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:54.507344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:54.574056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 566/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:54.846617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d3b3b-025e-4f24-ad78-2fcc2f92a34b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:54.946904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d3b3b-025e-4f24-ad78-2fcc2f92a34b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:54.948407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:54.980625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1d0de-ffb8-460a-abf2-e791833d0337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:54.981416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1d0de-ffb8-460a-abf2-e791833d0337 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:54.981740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1d0de-ffb8-460a-abf2-e791833d0337 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:54.982218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1d0de-ffb8-460a-abf2-e791833d0337 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:54.983801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1d0de-ffb8-460a-abf2-e791833d0337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:54.985235 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1d0de-ffb8-460a-abf2-e791833d0337 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:54.986841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 543/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:06:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:54.992599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc95fad-3382-4737-8b2e-7363f402ce58 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:55.090349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc95fad-3382-4737-8b2e-7363f402ce58 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:55.124138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39cdfb7-fc06-4b82-b0e3-6c09c2b102fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:55.124670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39cdfb7-fc06-4b82-b0e3-6c09c2b102fc None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:55.124949 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39cdfb7-fc06-4b82-b0e3-6c09c2b102fc None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:55.125218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39cdfb7-fc06-4b82-b0e3-6c09c2b102fc None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:55.125854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39cdfb7-fc06-4b82-b0e3-6c09c2b102fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:55.127302 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39cdfb7-fc06-4b82-b0e3-6c09c2b102fc None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:55.136942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:55.240438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 567/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:06:54 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:06:57.188460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94e0e7-2b61-4942-9b8d-519583dc927c None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:57.188993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94e0e7-2b61-4942-9b8d-519583dc927c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:57.188993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94e0e7-2b61-4942-9b8d-519583dc927c None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:57.190331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 544/1111] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:06:57.195650 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51822b-d15b-47f8-b5c9-76bb7ea04785 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:57.196029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51822b-d15b-47f8-b5c9-76bb7ea04785 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:57.196201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51822b-d15b-47f8-b5c9-76bb7ea04785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:57.213952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2d51822b-d15b-47f8-b5c9-76bb7ea04785 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:06:57.215712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2d51822b-d15b-47f8-b5c9-76bb7ea04785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:06:57.281341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 568/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:06:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:06:57.551268 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94e0e7-2b61-4942-9b8d-519583dc927c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:57.651037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94e0e7-2b61-4942-9b8d-519583dc927c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:57.685581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23748b9e-2ec0-4d23-9683-4ae260da1b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:57.686192 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23748b9e-2ec0-4d23-9683-4ae260da1b1c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:57.686464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23748b9e-2ec0-4d23-9683-4ae260da1b1c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:57.686762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23748b9e-2ec0-4d23-9683-4ae260da1b1c None admin] PATH_INFO: `/v3/roles/c2fe89b1b9fa47b19e02efd64ace9000` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:57.688154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23748b9e-2ec0-4d23-9683-4ae260da1b1c None admin] RBAC: Authorizing `identity:get_role(role_id=c2fe89b1b9fa47b19e02efd64ace9000)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:57.689488 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23748b9e-2ec0-4d23-9683-4ae260da1b1c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:57.690839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 545/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:06:57 2022] GET /identity/v3/roles/c2fe89b1b9fa47b19e02efd64ace9000 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:57.696660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51822b-d15b-47f8-b5c9-76bb7ea04785 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:57.792552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51822b-d15b-47f8-b5c9-76bb7ea04785 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:57.815813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:57.826045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207d03c-5418-45ae-85d7-369be76c9178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:57.829309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207d03c-5418-45ae-85d7-369be76c9178 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:57.829309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207d03c-5418-45ae-85d7-369be76c9178 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:57.829309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207d03c-5418-45ae-85d7-369be76c9178 None admin] PATH_INFO: `/v3/users/3ea31281a0554e39922bae2ddd45fc74` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:57.841090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207d03c-5418-45ae-85d7-369be76c9178 None admin] RBAC: Authorizing `identity:get_user(user_id=3ea31281a0554e39922bae2ddd45fc74)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:57.842599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207d03c-5418-45ae-85d7-369be76c9178 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:57.844195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 569/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:06:57 2022] GET /identity/v3/users/3ea31281a0554e39922bae2ddd45fc74 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:57.849994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23748b9e-2ec0-4d23-9683-4ae260da1b1c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:57.890298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:57.948245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23748b9e-2ec0-4d23-9683-4ae260da1b1c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:57.982674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb884d-3641-4754-999c-e405532be9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:57.983266 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb884d-3641-4754-999c-e405532be9d7 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:57.983541 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb884d-3641-4754-999c-e405532be9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:57.983810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb884d-3641-4754-999c-e405532be9d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:58.049450 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb884d-3641-4754-999c-e405532be9d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:58.050910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb884d-3641-4754-999c-e405532be9d7 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:58.087826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:06:58.146138 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 546/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:06:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:06:58.152058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207d03c-5418-45ae-85d7-369be76c9178 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:58.251282 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207d03c-5418-45ae-85d7-369be76c9178 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:58.287090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e761f22-18f0-460e-b080-451ee6bd54a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:58.287703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761f22-18f0-460e-b080-451ee6bd54a3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:58.287974 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761f22-18f0-460e-b080-451ee6bd54a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:58.288238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761f22-18f0-460e-b080-451ee6bd54a3 None admin] PATH_INFO: `/v3/projects/af769008beb142f9ac1cb545b08acb1a` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:58.289325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e761f22-18f0-460e-b080-451ee6bd54a3 None admin] RBAC: Authorizing `identity:get_project(project_id=af769008beb142f9ac1cb545b08acb1a)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:58.290771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e761f22-18f0-460e-b080-451ee6bd54a3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:58.292635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 570/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:06:58 2022] GET /identity/v3/projects/af769008beb142f9ac1cb545b08acb1a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:06:58.298938 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb884d-3641-4754-999c-e405532be9d7 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:06:58.399832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb884d-3641-4754-999c-e405532be9d7 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:06:58.433479 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985ccfa-77e5-4570-bced-7a67c0c5fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:06:58.434792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985ccfa-77e5-4570-bced-7a67c0c5fde4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:06:58.435121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985ccfa-77e5-4570-bced-7a67c0c5fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:06:58.435391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985ccfa-77e5-4570-bced-7a67c0c5fde4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:06:58.436100 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985ccfa-77e5-4570-bced-7a67c0c5fde4 None admin] RBAC: Adding query filter params (role.id=c2fe89b1b9fa47b19e02efd64ace9000, scope.project.id=af769008beb142f9ac1cb545b08acb1a, user.id=3ea31281a0554e39922bae2ddd45fc74) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:06:58.436403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985ccfa-77e5-4570-bced-7a67c0c5fde4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:06:58.437821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985ccfa-77e5-4570-bced-7a67c0c5fde4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:06:58.443074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 547/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:06:58 2022] GET /identity/v3/role_assignments?user.id=3ea31281a0554e39922bae2ddd45fc74&scope.project.id=af769008beb142f9ac1cb545b08acb1a&role.id=c2fe89b1b9fa47b19e02efd64ace9000 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:00.328147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db3ff0-1331-4761-8fd6-48e66cfd2913 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:00.328692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db3ff0-1331-4761-8fd6-48e66cfd2913 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:00.328692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db3ff0-1331-4761-8fd6-48e66cfd2913 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:00.330591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 571/1118] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:00.335854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f31ce-ac26-4f92-a2c0-54f032e431ca None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:00.336330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f31ce-ac26-4f92-a2c0-54f032e431ca None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:00.336441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f31ce-ac26-4f92-a2c0-54f032e431ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:00.342772 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:00.357963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c26f31ce-ac26-4f92-a2c0-54f032e431ca None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:00.359218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c26f31ce-ac26-4f92-a2c0-54f032e431ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:00.425468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 548/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:07:00.694164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db3ff0-1331-4761-8fd6-48e66cfd2913 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:00.744718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:00.795163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db3ff0-1331-4761-8fd6-48e66cfd2913 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:00.827354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8af46e-276f-41be-8630-6c936c79f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:00.827967 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8af46e-276f-41be-8630-6c936c79f60a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:00.828242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8af46e-276f-41be-8630-6c936c79f60a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:00.828510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8af46e-276f-41be-8630-6c936c79f60a None admin] PATH_INFO: `/v3/roles/c2fe89b1b9fa47b19e02efd64ace9000` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:00.829913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8af46e-276f-41be-8630-6c936c79f60a None admin] RBAC: Authorizing `identity:get_role(role_id=c2fe89b1b9fa47b19e02efd64ace9000)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:00.831304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8af46e-276f-41be-8630-6c936c79f60a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:00.832651 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 572/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:07:00 2022] GET /identity/v3/roles/c2fe89b1b9fa47b19e02efd64ace9000 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:00.838520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f31ce-ac26-4f92-a2c0-54f032e431ca None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:00.935189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f31ce-ac26-4f92-a2c0-54f032e431ca None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:00.963277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:00.968662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141f636-2fbf-4ed6-92d2-4a9ef4203666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:00.969280 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141f636-2fbf-4ed6-92d2-4a9ef4203666 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:00.969554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141f636-2fbf-4ed6-92d2-4a9ef4203666 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:00.969821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141f636-2fbf-4ed6-92d2-4a9ef4203666 None admin] PATH_INFO: `/v3/users/3ea31281a0554e39922bae2ddd45fc74` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:00.970966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141f636-2fbf-4ed6-92d2-4a9ef4203666 None admin] RBAC: Authorizing `identity:get_user(user_id=3ea31281a0554e39922bae2ddd45fc74)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:00.972390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141f636-2fbf-4ed6-92d2-4a9ef4203666 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:00.973958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 549/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:07:00 2022] GET /identity/v3/users/3ea31281a0554e39922bae2ddd45fc74 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:00.979823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8af46e-276f-41be-8630-6c936c79f60a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:01.079512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8af46e-276f-41be-8630-6c936c79f60a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:01.098009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:01.114893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343f3a2-65d6-4545-9d78-464e27eb943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:01.115517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f3a2-65d6-4545-9d78-464e27eb943c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:01.115791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f3a2-65d6-4545-9d78-464e27eb943c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:01.116058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f3a2-65d6-4545-9d78-464e27eb943c None admin] PATH_INFO: `/v3/projects/af769008beb142f9ac1cb545b08acb1a` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:01.117145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343f3a2-65d6-4545-9d78-464e27eb943c None admin] RBAC: Authorizing `identity:get_project(project_id=af769008beb142f9ac1cb545b08acb1a)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:01.118635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343f3a2-65d6-4545-9d78-464e27eb943c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:01.120254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 573/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:07:00 2022] GET /identity/v3/projects/af769008beb142f9ac1cb545b08acb1a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:01.126256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141f636-2fbf-4ed6-92d2-4a9ef4203666 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:01.226011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141f636-2fbf-4ed6-92d2-4a9ef4203666 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:01.258664 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bd06d-cd72-465d-90e4-a459336fa1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:01.259317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bd06d-cd72-465d-90e4-a459336fa1ed None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:01.259596 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bd06d-cd72-465d-90e4-a459336fa1ed None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:01.259866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bd06d-cd72-465d-90e4-a459336fa1ed None admin] PATH_INFO: `/v3/projects/af769008beb142f9ac1cb545b08acb1a/users/3ea31281a0554e39922bae2ddd45fc74/roles/c2fe89b1b9fa47b19e02efd64ace9000` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:01.262635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bd06d-cd72-465d-90e4-a459336fa1ed None admin] RBAC: Authorizing `identity:create_grant(project_id=af769008beb142f9ac1cb545b08acb1a, user_id=3ea31281a0554e39922bae2ddd45fc74, role_id=c2fe89b1b9fa47b19e02efd64ace9000)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:01.264174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bd06d-cd72-465d-90e4-a459336fa1ed None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:01.391522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 550/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:07:01 2022] PUT /identity/v3/projects/af769008beb142f9ac1cb545b08acb1a/users/3ea31281a0554e39922bae2ddd45fc74/roles/c2fe89b1b9fa47b19e02efd64ace9000 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:07:03.288350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a0a28-a052-4add-b662-91f3d85d0691 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:03.288916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a0a28-a052-4add-b662-91f3d85d0691 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:03.288916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a0a28-a052-4add-b662-91f3d85d0691 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:03.290233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 574/1124] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:03.296759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be60bd-41fb-4044-9441-baf74732d82a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:03.297220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be60bd-41fb-4044-9441-baf74732d82a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:03.297319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be60bd-41fb-4044-9441-baf74732d82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:03.315381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-e0be60bd-41fb-4044-9441-baf74732d82a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:03.316586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e0be60bd-41fb-4044-9441-baf74732d82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:03.373923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:03.384744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 551/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:03.663793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a0a28-a052-4add-b662-91f3d85d0691 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:03.764063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a0a28-a052-4add-b662-91f3d85d0691 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:03.796670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a794c0-d6f4-4402-a5b1-7aa40b0986b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:03.797282 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a794c0-d6f4-4402-a5b1-7aa40b0986b6 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:03.797554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a794c0-d6f4-4402-a5b1-7aa40b0986b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:03.797820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a794c0-d6f4-4402-a5b1-7aa40b0986b6 None admin] PATH_INFO: `/v3/roles/c2fe89b1b9fa47b19e02efd64ace9000` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:03.799212 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a794c0-d6f4-4402-a5b1-7aa40b0986b6 None admin] RBAC: Authorizing `identity:get_role(role_id=c2fe89b1b9fa47b19e02efd64ace9000)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:03.800577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a794c0-d6f4-4402-a5b1-7aa40b0986b6 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:03.801890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 575/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:07:03 2022] GET /identity/v3/roles/c2fe89b1b9fa47b19e02efd64ace9000 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:03.808269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be60bd-41fb-4044-9441-baf74732d82a None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:03.906902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be60bd-41fb-4044-9441-baf74732d82a None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:03.940770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84999dd6-2520-4dc8-8f56-607cc00858e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:03.941399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999dd6-2520-4dc8-8f56-607cc00858e5 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:03.941673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999dd6-2520-4dc8-8f56-607cc00858e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:03.941941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999dd6-2520-4dc8-8f56-607cc00858e5 None admin] PATH_INFO: `/v3/users/3ea31281a0554e39922bae2ddd45fc74` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:03.943099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84999dd6-2520-4dc8-8f56-607cc00858e5 None admin] RBAC: Authorizing `identity:get_user(user_id=3ea31281a0554e39922bae2ddd45fc74)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:03.944527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84999dd6-2520-4dc8-8f56-607cc00858e5 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:03.946107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 552/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:07:03 2022] GET /identity/v3/users/3ea31281a0554e39922bae2ddd45fc74 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:03.952942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a794c0-d6f4-4402-a5b1-7aa40b0986b6 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:04.006945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:04.050966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a794c0-d6f4-4402-a5b1-7aa40b0986b6 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:04.084826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27615d23-42e3-46d2-a485-d45085aa9fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:04.085510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27615d23-42e3-46d2-a485-d45085aa9fbe None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:04.085788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27615d23-42e3-46d2-a485-d45085aa9fbe None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:04.086055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27615d23-42e3-46d2-a485-d45085aa9fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:04.151243 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27615d23-42e3-46d2-a485-d45085aa9fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:04.152743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27615d23-42e3-46d2-a485-d45085aa9fbe None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:04.205837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:04.250535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 576/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:07:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:07:04.256804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84999dd6-2520-4dc8-8f56-607cc00858e5 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:04.315901 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:04.361218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84999dd6-2520-4dc8-8f56-607cc00858e5 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:04.395349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd5d71-ef2f-4faf-a2ac-f4e78247621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:04.395962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5d71-ef2f-4faf-a2ac-f4e78247621c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:04.396239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5d71-ef2f-4faf-a2ac-f4e78247621c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:04.396510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5d71-ef2f-4faf-a2ac-f4e78247621c None admin] PATH_INFO: `/v3/projects/af769008beb142f9ac1cb545b08acb1a` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:04.397614 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd5d71-ef2f-4faf-a2ac-f4e78247621c None admin] RBAC: Authorizing `identity:get_project(project_id=af769008beb142f9ac1cb545b08acb1a)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:04.399078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd5d71-ef2f-4faf-a2ac-f4e78247621c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:04.400713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 553/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:07:04 2022] GET /identity/v3/projects/af769008beb142f9ac1cb545b08acb1a => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:04.408347 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27615d23-42e3-46d2-a485-d45085aa9fbe None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:04.510090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27615d23-42e3-46d2-a485-d45085aa9fbe None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:04.544893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06df92-67b8-4f1b-a02c-cbb166eb745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:04.545494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06df92-67b8-4f1b-a02c-cbb166eb745e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:04.545772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06df92-67b8-4f1b-a02c-cbb166eb745e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:04.546042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06df92-67b8-4f1b-a02c-cbb166eb745e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:04.546792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06df92-67b8-4f1b-a02c-cbb166eb745e None admin] RBAC: Adding query filter params (role.id=c2fe89b1b9fa47b19e02efd64ace9000, scope.project.id=af769008beb142f9ac1cb545b08acb1a, user.id=3ea31281a0554e39922bae2ddd45fc74) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:04.547100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06df92-67b8-4f1b-a02c-cbb166eb745e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:04.548525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06df92-67b8-4f1b-a02c-cbb166eb745e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:04.554294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 577/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:07:04 2022] GET /identity/v3/role_assignments?user.id=3ea31281a0554e39922bae2ddd45fc74&scope.project.id=af769008beb142f9ac1cb545b08acb1a&role.id=c2fe89b1b9fa47b19e02efd64ace9000 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:06.502042 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441b3b9-efdb-409c-a24e-5c2f70d78c15 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:06.502606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441b3b9-efdb-409c-a24e-5c2f70d78c15 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:06.502606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441b3b9-efdb-409c-a24e-5c2f70d78c15 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:06.503961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 554/1131] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:06.509247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8628d-ea45-42e5-86e8-73b20ae75897 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:06.509681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8628d-ea45-42e5-86e8-73b20ae75897 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:06.509783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8628d-ea45-42e5-86e8-73b20ae75897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:06.528679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-46a8628d-ea45-42e5-86e8-73b20ae75897 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:06.529848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-46a8628d-ea45-42e5-86e8-73b20ae75897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:06.543478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:06.600299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 578/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:06.868137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441b3b9-efdb-409c-a24e-5c2f70d78c15 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:06.970983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441b3b9-efdb-409c-a24e-5c2f70d78c15 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:06.995473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:07.004324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887da475-f0e2-40f5-b9ca-d6fb08754da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:07.004945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887da475-f0e2-40f5-b9ca-d6fb08754da1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:07.005223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887da475-f0e2-40f5-b9ca-d6fb08754da1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:07.005494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887da475-f0e2-40f5-b9ca-d6fb08754da1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:07.006660 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887da475-f0e2-40f5-b9ca-d6fb08754da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:07.008095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887da475-f0e2-40f5-b9ca-d6fb08754da1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:07.012087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 555/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:07:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:07.016523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8628d-ea45-42e5-86e8-73b20ae75897 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:07.114091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8628d-ea45-42e5-86e8-73b20ae75897 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:07.147511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562c882-f145-4416-a2bf-2406dcf41cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:07.148105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562c882-f145-4416-a2bf-2406dcf41cdc None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:07.148378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562c882-f145-4416-a2bf-2406dcf41cdc None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:07.148641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562c882-f145-4416-a2bf-2406dcf41cdc None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:07.149290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562c882-f145-4416-a2bf-2406dcf41cdc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:07.150743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562c882-f145-4416-a2bf-2406dcf41cdc None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:07.259575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:07.266593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 579/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:07:07 2022] POST /identity/v3/projects => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:07:09.221575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52cb50-a782-4c49-a2e0-72d1e5f56d04 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:09.222134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52cb50-a782-4c49-a2e0-72d1e5f56d04 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:09.222134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52cb50-a782-4c49-a2e0-72d1e5f56d04 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:09.223526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 556/1135] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:09.228844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7d6bb-bd06-449e-a8e2-0eee5716c813 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:09.229228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7d6bb-bd06-449e-a8e2-0eee5716c813 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:09.229389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7d6bb-bd06-449e-a8e2-0eee5716c813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:09.247834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-a9e7d6bb-bd06-449e-a8e2-0eee5716c813 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:09.249016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a9e7d6bb-bd06-449e-a8e2-0eee5716c813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:09.315560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 580/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:09.588070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52cb50-a782-4c49-a2e0-72d1e5f56d04 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:09.690234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52cb50-a782-4c49-a2e0-72d1e5f56d04 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:09.723612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ea696-043f-4044-8ef0-c7be3b2b6406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:09.724229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ea696-043f-4044-8ef0-c7be3b2b6406 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:09.724502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ea696-043f-4044-8ef0-c7be3b2b6406 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:09.724768 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ea696-043f-4044-8ef0-c7be3b2b6406 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:09.725892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ea696-043f-4044-8ef0-c7be3b2b6406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:09.727344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ea696-043f-4044-8ef0-c7be3b2b6406 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:09.728901 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 557/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:07:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:09.735365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7d6bb-bd06-449e-a8e2-0eee5716c813 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:09.831883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7d6bb-bd06-449e-a8e2-0eee5716c813 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:09.860527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:09.866095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312eef7-0113-4407-800c-cccdda5d633f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:09.866730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312eef7-0113-4407-800c-cccdda5d633f None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:09.867010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312eef7-0113-4407-800c-cccdda5d633f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:09.867291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312eef7-0113-4407-800c-cccdda5d633f None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:09.874517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312eef7-0113-4407-800c-cccdda5d633f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:09.874517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312eef7-0113-4407-800c-cccdda5d633f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:09.984513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 581/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:07:09 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:07:11.942712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fe1a2-4d1d-41b8-b646-83ec574b1c9d None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:11.943337 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fe1a2-4d1d-41b8-b646-83ec574b1c9d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:11.943337 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fe1a2-4d1d-41b8-b646-83ec574b1c9d None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:11.944610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 558/1139] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:11.950841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960812-b15b-4d8f-a286-3fe69ecfb264 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:11.951281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960812-b15b-4d8f-a286-3fe69ecfb264 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:11.951380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960812-b15b-4d8f-a286-3fe69ecfb264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:11.969050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-54960812-b15b-4d8f-a286-3fe69ecfb264 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:11.970230 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-54960812-b15b-4d8f-a286-3fe69ecfb264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:12.035656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 582/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:12.316088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fe1a2-4d1d-41b8-b646-83ec574b1c9d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:12.377848 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:12.415879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fe1a2-4d1d-41b8-b646-83ec574b1c9d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:12.452067 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d87cc-bf2e-4986-880f-ea8f3a9fcefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:12.452741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d87cc-bf2e-4986-880f-ea8f3a9fcefe None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:12.453051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d87cc-bf2e-4986-880f-ea8f3a9fcefe None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:12.453356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d87cc-bf2e-4986-880f-ea8f3a9fcefe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:12.463861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d87cc-bf2e-4986-880f-ea8f3a9fcefe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-479d87cc-bf2e-4986-880f-ea8f3a9fcefe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:07:12.468907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:07:12.470082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:07:12.471390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 559/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:07:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:07:12.476081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54960812-b15b-4d8f-a286-3fe69ecfb264 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:12.576011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54960812-b15b-4d8f-a286-3fe69ecfb264 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:12.592586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:12.610750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539656b-3f57-48b6-8b44-07022b1e5302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:12.611587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539656b-3f57-48b6-8b44-07022b1e5302 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:12.611868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539656b-3f57-48b6-8b44-07022b1e5302 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:12.612143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539656b-3f57-48b6-8b44-07022b1e5302 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:12.612848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539656b-3f57-48b6-8b44-07022b1e5302 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:12.613146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539656b-3f57-48b6-8b44-07022b1e5302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:12.614578 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539656b-3f57-48b6-8b44-07022b1e5302 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:12.623788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 583/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:07:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:12.629662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d87cc-bf2e-4986-880f-ea8f3a9fcefe None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:12.731171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d87cc-bf2e-4986-880f-ea8f3a9fcefe None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:12.753232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:12.765033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e088ad-f485-46f0-9993-964340002983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:12.769542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e088ad-f485-46f0-9993-964340002983 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:12.769542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e088ad-f485-46f0-9993-964340002983 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:12.769542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e088ad-f485-46f0-9993-964340002983 None admin] PATH_INFO: `/v3/users/1ca5478bd2284d6fadd602b9204498ab` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:12.779677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e088ad-f485-46f0-9993-964340002983 None admin] RBAC: Authorizing `identity:get_user(user_id=1ca5478bd2284d6fadd602b9204498ab)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:12.781138 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e088ad-f485-46f0-9993-964340002983 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:12.782785 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 560/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:07:12 2022] GET /identity/v3/users/1ca5478bd2284d6fadd602b9204498ab => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:12.789117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539656b-3f57-48b6-8b44-07022b1e5302 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:12.887500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539656b-3f57-48b6-8b44-07022b1e5302 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:12.921114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1218f-5c58-4e8d-a00e-5347d7738daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:12.921712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1218f-5c58-4e8d-a00e-5347d7738daf None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:12.921997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1218f-5c58-4e8d-a00e-5347d7738daf None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:12.922269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1218f-5c58-4e8d-a00e-5347d7738daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:12.967680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:12.988937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1218f-5c58-4e8d-a00e-5347d7738daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:12.990398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1218f-5c58-4e8d-a00e-5347d7738daf None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:13.086248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 584/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:07:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:07:13.092954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e088ad-f485-46f0-9993-964340002983 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:13.207101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e088ad-f485-46f0-9993-964340002983 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:13.241434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4d9ac-dd3d-4b7e-8c0e-1456e706ea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:13.242038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d9ac-dd3d-4b7e-8c0e-1456e706ea57 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:13.242309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d9ac-dd3d-4b7e-8c0e-1456e706ea57 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:13.242603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d9ac-dd3d-4b7e-8c0e-1456e706ea57 None admin] PATH_INFO: `/v3/projects/8f3b3f3547e04b16a2cfe40dbdd096f6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:13.243684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4d9ac-dd3d-4b7e-8c0e-1456e706ea57 None admin] RBAC: Authorizing `identity:get_project(project_id=8f3b3f3547e04b16a2cfe40dbdd096f6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:13.245093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4d9ac-dd3d-4b7e-8c0e-1456e706ea57 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:13.247040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 561/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:07:13 2022] GET /identity/v3/projects/8f3b3f3547e04b16a2cfe40dbdd096f6 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:13.253229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1218f-5c58-4e8d-a00e-5347d7738daf None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:13.330163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:13.352119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1218f-5c58-4e8d-a00e-5347d7738daf None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:13.385576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156805b4-8460-44e0-897e-48ad72b4dbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:13.386793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156805b4-8460-44e0-897e-48ad72b4dbbd None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:13.387198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156805b4-8460-44e0-897e-48ad72b4dbbd None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:13.387472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156805b4-8460-44e0-897e-48ad72b4dbbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:13.388190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156805b4-8460-44e0-897e-48ad72b4dbbd None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=8f3b3f3547e04b16a2cfe40dbdd096f6, user.id=1ca5478bd2284d6fadd602b9204498ab) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:13.388542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156805b4-8460-44e0-897e-48ad72b4dbbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:13.389971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156805b4-8460-44e0-897e-48ad72b4dbbd None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:13.395339 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 585/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:07:13 2022] GET /identity/v3/role_assignments?user.id=1ca5478bd2284d6fadd602b9204498ab&scope.project.id=8f3b3f3547e04b16a2cfe40dbdd096f6&role.id=4b10782d99b74f5bb453b82286012e0c => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:15.303931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba6aa1-c6ae-43ea-94b8-db6d08c9c003 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:15.304505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba6aa1-c6ae-43ea-94b8-db6d08c9c003 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:15.304505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba6aa1-c6ae-43ea-94b8-db6d08c9c003 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:15.305812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 562/1147] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:15.311674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a79d-f048-4a5c-b704-c787403ea715 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:15.312130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a79d-f048-4a5c-b704-c787403ea715 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:15.312231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a79d-f048-4a5c-b704-c787403ea715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:15.332233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1178a79d-f048-4a5c-b704-c787403ea715 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:15.333418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1178a79d-f048-4a5c-b704-c787403ea715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:15.403518 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 586/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:07:15.673759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba6aa1-c6ae-43ea-94b8-db6d08c9c003 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:15.724272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:15.774936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba6aa1-c6ae-43ea-94b8-db6d08c9c003 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:15.808938 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee5e74-45fd-46f4-92fb-d74203ada3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:15.809544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee5e74-45fd-46f4-92fb-d74203ada3d4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:15.809820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee5e74-45fd-46f4-92fb-d74203ada3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:15.810089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee5e74-45fd-46f4-92fb-d74203ada3d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:15.819880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee5e74-45fd-46f4-92fb-d74203ada3d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-e9ee5e74-45fd-46f4-92fb-d74203ada3d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:07:15.824584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:07:15.825696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:07:15.826777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 563/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:07:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:07:15.831871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178a79d-f048-4a5c-b704-c787403ea715 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:15.900012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:15.928872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178a79d-f048-4a5c-b704-c787403ea715 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:15.961506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd582b2-6a4e-4440-ab9c-dc4e5b889ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:15.962109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd582b2-6a4e-4440-ab9c-dc4e5b889ac3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:15.962386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd582b2-6a4e-4440-ab9c-dc4e5b889ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:15.962687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd582b2-6a4e-4440-ab9c-dc4e5b889ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:15.963341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd582b2-6a4e-4440-ab9c-dc4e5b889ac3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:15.963642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd582b2-6a4e-4440-ab9c-dc4e5b889ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:15.965684 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd582b2-6a4e-4440-ab9c-dc4e5b889ac3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:15.973498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 587/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:07:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:15.979425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee5e74-45fd-46f4-92fb-d74203ada3d4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:16.077705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee5e74-45fd-46f4-92fb-d74203ada3d4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:16.086663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:16.111129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddc958-f042-479c-946a-4969962d1cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:16.111751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddc958-f042-479c-946a-4969962d1cdd None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:16.112027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddc958-f042-479c-946a-4969962d1cdd None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:16.112297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddc958-f042-479c-946a-4969962d1cdd None admin] PATH_INFO: `/v3/users/1ca5478bd2284d6fadd602b9204498ab` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:16.113453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddc958-f042-479c-946a-4969962d1cdd None admin] RBAC: Authorizing `identity:get_user(user_id=1ca5478bd2284d6fadd602b9204498ab)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:16.114972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddc958-f042-479c-946a-4969962d1cdd None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:16.116780 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 564/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:07:15 2022] GET /identity/v3/users/1ca5478bd2284d6fadd602b9204498ab => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:16.122635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd582b2-6a4e-4440-ab9c-dc4e5b889ac3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:16.221838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd582b2-6a4e-4440-ab9c-dc4e5b889ac3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:16.253697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c149c-fbfd-476d-94ef-0390afd6c1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:16.254320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c149c-fbfd-476d-94ef-0390afd6c1df None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:16.254627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c149c-fbfd-476d-94ef-0390afd6c1df None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:16.254901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c149c-fbfd-476d-94ef-0390afd6c1df None admin] PATH_INFO: `/v3/projects/8f3b3f3547e04b16a2cfe40dbdd096f6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:16.256005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c149c-fbfd-476d-94ef-0390afd6c1df None admin] RBAC: Authorizing `identity:get_project(project_id=8f3b3f3547e04b16a2cfe40dbdd096f6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:16.257438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c149c-fbfd-476d-94ef-0390afd6c1df None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:16.259096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 588/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:07:16 2022] GET /identity/v3/projects/8f3b3f3547e04b16a2cfe40dbdd096f6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:16.265804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddc958-f042-479c-946a-4969962d1cdd None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:16.363852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddc958-f042-479c-946a-4969962d1cdd None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:16.397195 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036aaafe-e289-4ae7-a623-c76d21ed0e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:16.397841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036aaafe-e289-4ae7-a623-c76d21ed0e42 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:16.398119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036aaafe-e289-4ae7-a623-c76d21ed0e42 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:16.398396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036aaafe-e289-4ae7-a623-c76d21ed0e42 None admin] PATH_INFO: `/v3/projects/8f3b3f3547e04b16a2cfe40dbdd096f6/users/1ca5478bd2284d6fadd602b9204498ab/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:16.400290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036aaafe-e289-4ae7-a623-c76d21ed0e42 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f3b3f3547e04b16a2cfe40dbdd096f6, user_id=1ca5478bd2284d6fadd602b9204498ab, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:16.402347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036aaafe-e289-4ae7-a623-c76d21ed0e42 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:16.492559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 565/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:07:16 2022] PUT /identity/v3/projects/8f3b3f3547e04b16a2cfe40dbdd096f6/users/1ca5478bd2284d6fadd602b9204498ab/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:07:18.388508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428630c6-e568-4663-a682-6484abeef7a6 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:18.389062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428630c6-e568-4663-a682-6484abeef7a6 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:18.389062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428630c6-e568-4663-a682-6484abeef7a6 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:18.390377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 589/1154] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:18.395749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858cd87-0a4a-4cfa-8452-7af6fd88f9a1 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:18.396164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858cd87-0a4a-4cfa-8452-7af6fd88f9a1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:18.396287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858cd87-0a4a-4cfa-8452-7af6fd88f9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:18.414753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-0858cd87-0a4a-4cfa-8452-7af6fd88f9a1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:18.415931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0858cd87-0a4a-4cfa-8452-7af6fd88f9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:18.444096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:18.482614 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 566/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:18.754383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428630c6-e568-4663-a682-6484abeef7a6 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:18.757382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:18.858808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428630c6-e568-4663-a682-6484abeef7a6 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:18.892721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7569c-1806-4b32-bd99-e17346873a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:18.893333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7569c-1806-4b32-bd99-e17346873a17 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:18.893606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7569c-1806-4b32-bd99-e17346873a17 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:18.893876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7569c-1806-4b32-bd99-e17346873a17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:18.904521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7569c-1806-4b32-bd99-e17346873a17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-2ce7569c-1806-4b32-bd99-e17346873a17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:07:18.909070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:07:18.910180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:07:18.911283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 590/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:07:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:07:18.916493 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858cd87-0a4a-4cfa-8452-7af6fd88f9a1 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:19.013235 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858cd87-0a4a-4cfa-8452-7af6fd88f9a1 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:19.047395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf6d85-b5dd-409d-bc79-4dfc2b09f399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:19.047992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf6d85-b5dd-409d-bc79-4dfc2b09f399 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:19.048267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf6d85-b5dd-409d-bc79-4dfc2b09f399 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:19.048537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf6d85-b5dd-409d-bc79-4dfc2b09f399 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:19.049198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf6d85-b5dd-409d-bc79-4dfc2b09f399 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:19.049496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf6d85-b5dd-409d-bc79-4dfc2b09f399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:19.050889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf6d85-b5dd-409d-bc79-4dfc2b09f399 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:19.058698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 567/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:19.064478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7569c-1806-4b32-bd99-e17346873a17 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:19.124158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:19.162179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7569c-1806-4b32-bd99-e17346873a17 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:19.196112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb010c1-7c8e-4d3e-9d79-1b7f0dd4ff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:19.196728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb010c1-7c8e-4d3e-9d79-1b7f0dd4ff00 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:19.197001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb010c1-7c8e-4d3e-9d79-1b7f0dd4ff00 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:19.197273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb010c1-7c8e-4d3e-9d79-1b7f0dd4ff00 None admin] PATH_INFO: `/v3/users/1ca5478bd2284d6fadd602b9204498ab` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:19.198920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb010c1-7c8e-4d3e-9d79-1b7f0dd4ff00 None admin] RBAC: Authorizing `identity:get_user(user_id=1ca5478bd2284d6fadd602b9204498ab)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:19.200655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb010c1-7c8e-4d3e-9d79-1b7f0dd4ff00 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:19.202222 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 591/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:07:19 2022] GET /identity/v3/users/1ca5478bd2284d6fadd602b9204498ab => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:19.208377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf6d85-b5dd-409d-bc79-4dfc2b09f399 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:19.237206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:19.308635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf6d85-b5dd-409d-bc79-4dfc2b09f399 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:19.343100 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3b6a8-fb6f-476f-b6dc-f8a0942b9066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:19.343695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3b6a8-fb6f-476f-b6dc-f8a0942b9066 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:19.343968 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3b6a8-fb6f-476f-b6dc-f8a0942b9066 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:19.344232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3b6a8-fb6f-476f-b6dc-f8a0942b9066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:19.414111 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3b6a8-fb6f-476f-b6dc-f8a0942b9066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:19.415603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3b6a8-fb6f-476f-b6dc-f8a0942b9066 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:19.442594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:19.514873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 568/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:07:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:07:19.521116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb010c1-7c8e-4d3e-9d79-1b7f0dd4ff00 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:19.621185 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb010c1-7c8e-4d3e-9d79-1b7f0dd4ff00 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:19.649468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:19.655253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cfcfd-9c2e-44a8-b47c-46d430691287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:19.655880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cfcfd-9c2e-44a8-b47c-46d430691287 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:19.656182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cfcfd-9c2e-44a8-b47c-46d430691287 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:19.656471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cfcfd-9c2e-44a8-b47c-46d430691287 None admin] PATH_INFO: `/v3/projects/8f3b3f3547e04b16a2cfe40dbdd096f6` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:19.657624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cfcfd-9c2e-44a8-b47c-46d430691287 None admin] RBAC: Authorizing `identity:get_project(project_id=8f3b3f3547e04b16a2cfe40dbdd096f6)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:19.659131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cfcfd-9c2e-44a8-b47c-46d430691287 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:19.660847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 592/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:07:19 2022] GET /identity/v3/projects/8f3b3f3547e04b16a2cfe40dbdd096f6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:19.667226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3b6a8-fb6f-476f-b6dc-f8a0942b9066 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:19.766140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3b6a8-fb6f-476f-b6dc-f8a0942b9066 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:19.791229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:19.799636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06ec37-1e73-4c33-8084-94cd76a5a405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:19.805117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06ec37-1e73-4c33-8084-94cd76a5a405 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:19.805117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06ec37-1e73-4c33-8084-94cd76a5a405 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:19.805117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06ec37-1e73-4c33-8084-94cd76a5a405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:19.805117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06ec37-1e73-4c33-8084-94cd76a5a405 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=8f3b3f3547e04b16a2cfe40dbdd096f6, user.id=1ca5478bd2284d6fadd602b9204498ab) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:19.805117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06ec37-1e73-4c33-8084-94cd76a5a405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:19.805117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06ec37-1e73-4c33-8084-94cd76a5a405 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:19.809586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 569/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:07:19 2022] GET /identity/v3/role_assignments?user.id=1ca5478bd2284d6fadd602b9204498ab&scope.project.id=8f3b3f3547e04b16a2cfe40dbdd096f6&role.id=4b10782d99b74f5bb453b82286012e0c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:07:21.740606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e803a9-aa92-499e-82ce-c9e8a58f649f None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:21.741129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e803a9-aa92-499e-82ce-c9e8a58f649f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:21.741129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e803a9-aa92-499e-82ce-c9e8a58f649f None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:21.742665 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 593/1162] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:21.747841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881807c-293a-4813-8154-d4048179e267 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:21.748246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881807c-293a-4813-8154-d4048179e267 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:21.748382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881807c-293a-4813-8154-d4048179e267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:21.766261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1881807c-293a-4813-8154-d4048179e267 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:21.767501 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1881807c-293a-4813-8154-d4048179e267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:21.834432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 570/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:22.105189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e803a9-aa92-499e-82ce-c9e8a58f649f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:22.205111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e803a9-aa92-499e-82ce-c9e8a58f649f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:22.238077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddf2d5-6675-4915-a2a2-03cf5d69a7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:22.238702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddf2d5-6675-4915-a2a2-03cf5d69a7e9 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:22.238985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddf2d5-6675-4915-a2a2-03cf5d69a7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:22.239257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddf2d5-6675-4915-a2a2-03cf5d69a7e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:22.302796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:22.306134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddf2d5-6675-4915-a2a2-03cf5d69a7e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:22.307625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddf2d5-6675-4915-a2a2-03cf5d69a7e9 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:22.406127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 594/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:07:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:07:22.412230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881807c-293a-4813-8154-d4048179e267 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:22.510502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881807c-293a-4813-8154-d4048179e267 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:22.535265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:22.547010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b566b-fb2b-4ee4-9d7b-9ac541dbcf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:22.548633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b566b-fb2b-4ee4-9d7b-9ac541dbcf79 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:22.548633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b566b-fb2b-4ee4-9d7b-9ac541dbcf79 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:22.548633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b566b-fb2b-4ee4-9d7b-9ac541dbcf79 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:22.554929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b566b-fb2b-4ee4-9d7b-9ac541dbcf79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:22.556373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b566b-fb2b-4ee4-9d7b-9ac541dbcf79 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-357b566b-fb2b-4ee4-9d7b-9ac541dbcf79 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:07:22.568695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:07:22.569922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:07:22.570872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 571/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 22 17:07:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:07:22.576823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddf2d5-6675-4915-a2a2-03cf5d69a7e9 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:22.675055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:22.678734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddf2d5-6675-4915-a2a2-03cf5d69a7e9 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:22.713663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfad24-b185-41c9-ad57-65b654d3fae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:22.714848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfad24-b185-41c9-ad57-65b654d3fae6 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:22.715319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfad24-b185-41c9-ad57-65b654d3fae6 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:22.715747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfad24-b185-41c9-ad57-65b654d3fae6 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:22.716701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfad24-b185-41c9-ad57-65b654d3fae6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:22.716999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfad24-b185-41c9-ad57-65b654d3fae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:22.718345 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfad24-b185-41c9-ad57-65b654d3fae6 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:22.725322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 595/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 22 17:07:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:24.644267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e520f3-1d80-46fd-b0c5-a14cbd5ced07 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:24.707115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e520f3-1d80-46fd-b0c5-a14cbd5ced07 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:24.707115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e520f3-1d80-46fd-b0c5-a14cbd5ced07 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:24.707115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 572/1167] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:24.707437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393b2f-a2cc-4fdd-bab3-d34ca0240311 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:24.707437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393b2f-a2cc-4fdd-bab3-d34ca0240311 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:24.707437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393b2f-a2cc-4fdd-bab3-d34ca0240311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:24.707437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-b1393b2f-a2cc-4fdd-bab3-d34ca0240311 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:24.707437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b1393b2f-a2cc-4fdd-bab3-d34ca0240311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:24.741157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 596/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:25.008354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e520f3-1d80-46fd-b0c5-a14cbd5ced07 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:25.109884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e520f3-1d80-46fd-b0c5-a14cbd5ced07 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:25.143018 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec25b51-8afc-4868-b4af-d8afbe53f049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:25.143612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec25b51-8afc-4868-b4af-d8afbe53f049 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:25.143893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec25b51-8afc-4868-b4af-d8afbe53f049 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:25.144166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec25b51-8afc-4868-b4af-d8afbe53f049 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:25.144833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec25b51-8afc-4868-b4af-d8afbe53f049 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:25.146199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec25b51-8afc-4868-b4af-d8afbe53f049 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:25.243010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 573/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Mon Aug 22 17:07:25 2022] POST /identity/v3/domains => generated 267 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:07:27.185574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069afa07-091a-48c3-85cf-57015becfc8e None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:27.186132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069afa07-091a-48c3-85cf-57015becfc8e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:27.186132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069afa07-091a-48c3-85cf-57015becfc8e None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:27.187510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 597/1170] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:27.192825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f0e9b-e46e-4466-b9d4-2926dee3df65 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:27.193254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f0e9b-e46e-4466-b9d4-2926dee3df65 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:27.193358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f0e9b-e46e-4466-b9d4-2926dee3df65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:27.211799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-904f0e9b-e46e-4466-b9d4-2926dee3df65 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:27.212996 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-904f0e9b-e46e-4466-b9d4-2926dee3df65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:27.247653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:27.278332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 574/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:07:27.547347 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069afa07-091a-48c3-85cf-57015becfc8e None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:27.639412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:27.648634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069afa07-091a-48c3-85cf-57015becfc8e None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:27.682118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00c642-d56b-4b94-a827-be57619129dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:27.683303 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00c642-d56b-4b94-a827-be57619129dc None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:27.683588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00c642-d56b-4b94-a827-be57619129dc None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:27.683858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00c642-d56b-4b94-a827-be57619129dc None admin] PATH_INFO: `/v3/domains/a4bc345b82d94793b956b8803e7d2507` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:27.693916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00c642-d56b-4b94-a827-be57619129dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=a4bc345b82d94793b956b8803e7d2507)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:27.695383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00c642-d56b-4b94-a827-be57619129dc None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:27.696969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 598/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Aug 22 17:07:27 2022] GET /identity/v3/domains/a4bc345b82d94793b956b8803e7d2507 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:27.703231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f0e9b-e46e-4466-b9d4-2926dee3df65 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:27.799613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f0e9b-e46e-4466-b9d4-2926dee3df65 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:27.832153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71726b0-feee-4395-aadb-a3c07d983f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:27.832752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71726b0-feee-4395-aadb-a3c07d983f81 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:27.833028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71726b0-feee-4395-aadb-a3c07d983f81 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:27.833297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71726b0-feee-4395-aadb-a3c07d983f81 None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:27.833944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71726b0-feee-4395-aadb-a3c07d983f81 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:27.835398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71726b0-feee-4395-aadb-a3c07d983f81 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:27.934294 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 575/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:07:27 2022] POST /identity/v3/projects => generated 361 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:07:29.869451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d612b-3e3d-4be1-a698-c79ec1b5753f None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:29.870010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d612b-3e3d-4be1-a698-c79ec1b5753f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:29.870010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d612b-3e3d-4be1-a698-c79ec1b5753f None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:29.872231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 599/1174] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:29.877460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b24825-d216-4a16-9ddd-4044d2beb587 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:29.877930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b24825-d216-4a16-9ddd-4044d2beb587 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:29.877930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b24825-d216-4a16-9ddd-4044d2beb587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:29.895989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-95b24825-d216-4a16-9ddd-4044d2beb587 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:29.897169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-95b24825-d216-4a16-9ddd-4044d2beb587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:29.914237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:29.963277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 576/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:30.235343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d612b-3e3d-4be1-a698-c79ec1b5753f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:30.336839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d612b-3e3d-4be1-a698-c79ec1b5753f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:30.375499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47654e8a-41bd-4f1a-8048-a25b780d3e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:30.376132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654e8a-41bd-4f1a-8048-a25b780d3e85 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:30.376406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654e8a-41bd-4f1a-8048-a25b780d3e85 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:30.376676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654e8a-41bd-4f1a-8048-a25b780d3e85 None admin] PATH_INFO: `/v3/domains/a4bc345b82d94793b956b8803e7d2507` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:30.377787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47654e8a-41bd-4f1a-8048-a25b780d3e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a4bc345b82d94793b956b8803e7d2507)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:30.379242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47654e8a-41bd-4f1a-8048-a25b780d3e85 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:30.380818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 600/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Aug 22 17:07:30 2022] GET /identity/v3/domains/a4bc345b82d94793b956b8803e7d2507 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:30.386965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b24825-d216-4a16-9ddd-4044d2beb587 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:30.486947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b24825-d216-4a16-9ddd-4044d2beb587 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:30.521251 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305db89-5a0c-4dea-ac63-badcd2f7033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:30.521851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305db89-5a0c-4dea-ac63-badcd2f7033e None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:30.522127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305db89-5a0c-4dea-ac63-badcd2f7033e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:30.522399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305db89-5a0c-4dea-ac63-badcd2f7033e None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:30.523065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305db89-5a0c-4dea-ac63-badcd2f7033e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:30.524484 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305db89-5a0c-4dea-ac63-badcd2f7033e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:30.693347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 577/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:07:30 2022] POST /identity/v3/users => generated 311 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:07:32.642971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f08f2f-32c4-450a-a5e7-d38cdf93001d None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:32.643478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f08f2f-32c4-450a-a5e7-d38cdf93001d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:32.643574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f08f2f-32c4-450a-a5e7-d38cdf93001d None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:32.644906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 601/1178] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:32.650261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b37e15-c2e3-4852-9393-369318434025 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:32.650752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b37e15-c2e3-4852-9393-369318434025 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:32.650873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b37e15-c2e3-4852-9393-369318434025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:32.669280 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-13b37e15-c2e3-4852-9393-369318434025 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:32.670553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-13b37e15-c2e3-4852-9393-369318434025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:32.673275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:32.739839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 578/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:33.017835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f08f2f-32c4-450a-a5e7-d38cdf93001d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:33.039873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:33.117952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f08f2f-32c4-450a-a5e7-d38cdf93001d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:33.151155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e909b-2e2c-4bbb-aaf4-4684f2c3b873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:33.151931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e909b-2e2c-4bbb-aaf4-4684f2c3b873 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:33.152314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e909b-2e2c-4bbb-aaf4-4684f2c3b873 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:33.152588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e909b-2e2c-4bbb-aaf4-4684f2c3b873 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:33.164089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e909b-2e2c-4bbb-aaf4-4684f2c3b873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-322e909b-2e2c-4bbb-aaf4-4684f2c3b873 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:07:33.169176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:07:33.170344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:07:33.175506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 602/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:07:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:07:33.176681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b37e15-c2e3-4852-9393-369318434025 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:33.275822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b37e15-c2e3-4852-9393-369318434025 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:33.310328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c247f3f-2f4f-4c38-a9f0-cc8fb24922be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:33.310971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c247f3f-2f4f-4c38-a9f0-cc8fb24922be None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:33.311251 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c247f3f-2f4f-4c38-a9f0-cc8fb24922be None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:33.311525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c247f3f-2f4f-4c38-a9f0-cc8fb24922be None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:33.312183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c247f3f-2f4f-4c38-a9f0-cc8fb24922be None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:33.312486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c247f3f-2f4f-4c38-a9f0-cc8fb24922be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:33.313870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c247f3f-2f4f-4c38-a9f0-cc8fb24922be None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:33.322089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 579/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:07:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:33.327959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e909b-2e2c-4bbb-aaf4-4684f2c3b873 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:33.414753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:33.435200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e909b-2e2c-4bbb-aaf4-4684f2c3b873 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:33.469761 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7bedc-7a85-4533-9fed-a70be970a894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:33.470380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7bedc-7a85-4533-9fed-a70be970a894 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:33.470688 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7bedc-7a85-4533-9fed-a70be970a894 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:33.471043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7bedc-7a85-4533-9fed-a70be970a894 None admin] PATH_INFO: `/v3/users/c6ed86e1f9624e22930369753605ad41` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:33.486131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7bedc-7a85-4533-9fed-a70be970a894 None admin] RBAC: Authorizing `identity:get_user(user_id=c6ed86e1f9624e22930369753605ad41)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:33.487749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7bedc-7a85-4533-9fed-a70be970a894 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:33.489394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 603/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:07:33 2022] GET /identity/v3/users/c6ed86e1f9624e22930369753605ad41 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:33.496253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c247f3f-2f4f-4c38-a9f0-cc8fb24922be None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:33.499230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:33.600231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c247f3f-2f4f-4c38-a9f0-cc8fb24922be None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:33.636518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff9794-3586-4730-90ec-3bc6224e7843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:33.637126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff9794-3586-4730-90ec-3bc6224e7843 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:33.637406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff9794-3586-4730-90ec-3bc6224e7843 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:33.637677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff9794-3586-4730-90ec-3bc6224e7843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:33.703835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff9794-3586-4730-90ec-3bc6224e7843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:33.705290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff9794-3586-4730-90ec-3bc6224e7843 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:33.707129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:33.804785 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 580/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:07:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:07:33.811228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7bedc-7a85-4533-9fed-a70be970a894 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:33.910781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7bedc-7a85-4533-9fed-a70be970a894 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:33.943879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119dcbb-abaf-4fd1-9674-305bbfed7aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:33.944493 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119dcbb-abaf-4fd1-9674-305bbfed7aa7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:33.944768 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119dcbb-abaf-4fd1-9674-305bbfed7aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:33.945038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119dcbb-abaf-4fd1-9674-305bbfed7aa7 None admin] PATH_INFO: `/v3/projects/d688d3b369c140378eac8fadaedb5470` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:33.946128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119dcbb-abaf-4fd1-9674-305bbfed7aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=d688d3b369c140378eac8fadaedb5470)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:33.947583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119dcbb-abaf-4fd1-9674-305bbfed7aa7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:33.949274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 604/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:07:33 2022] GET /identity/v3/projects/d688d3b369c140378eac8fadaedb5470 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:33.955476 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff9794-3586-4730-90ec-3bc6224e7843 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:34.054797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff9794-3586-4730-90ec-3bc6224e7843 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:34.056324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:34.089883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d34824-45b9-48b2-9b89-5169c12718f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:34.090498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34824-45b9-48b2-9b89-5169c12718f5 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:34.090788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34824-45b9-48b2-9b89-5169c12718f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:34.091062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34824-45b9-48b2-9b89-5169c12718f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:34.091785 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d34824-45b9-48b2-9b89-5169c12718f5 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=d688d3b369c140378eac8fadaedb5470, user.id=c6ed86e1f9624e22930369753605ad41) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:34.092086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d34824-45b9-48b2-9b89-5169c12718f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:34.093559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d34824-45b9-48b2-9b89-5169c12718f5 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:34.099204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 581/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:07:33 2022] GET /identity/v3/role_assignments?user.id=c6ed86e1f9624e22930369753605ad41&scope.project.id=d688d3b369c140378eac8fadaedb5470&role.id=4b10782d99b74f5bb453b82286012e0c => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:36.031982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f23fa-9c93-43a1-8331-3fdc9811491c None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:36.032545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f23fa-9c93-43a1-8331-3fdc9811491c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:36.032545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f23fa-9c93-43a1-8331-3fdc9811491c None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:36.033862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 605/1186] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:36.039148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104adf94-8fac-4aa3-9345-8f6f2c953f6f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:36.039598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104adf94-8fac-4aa3-9345-8f6f2c953f6f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:36.039705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104adf94-8fac-4aa3-9345-8f6f2c953f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:36.058915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-104adf94-8fac-4aa3-9345-8f6f2c953f6f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:36.060142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-104adf94-8fac-4aa3-9345-8f6f2c953f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:36.125411 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 582/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:36.407000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f23fa-9c93-43a1-8331-3fdc9811491c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:36.415945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:36.509698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f23fa-9c93-43a1-8331-3fdc9811491c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:36.544868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b9aca-3c74-4e0a-ae2e-a253d4cb8f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:36.545486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b9aca-3c74-4e0a-ae2e-a253d4cb8f2b None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:36.545767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b9aca-3c74-4e0a-ae2e-a253d4cb8f2b None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:36.546043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b9aca-3c74-4e0a-ae2e-a253d4cb8f2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:36.556349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b9aca-3c74-4e0a-ae2e-a253d4cb8f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-fa1b9aca-3c74-4e0a-ae2e-a253d4cb8f2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:07:36.561780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:07:36.569017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:07:36.570068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 606/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:07:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:07:36.571024 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104adf94-8fac-4aa3-9345-8f6f2c953f6f None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:36.665484 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:36.672188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104adf94-8fac-4aa3-9345-8f6f2c953f6f None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:36.705681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd14c6-c63a-4468-a31c-ef643398528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:36.706281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd14c6-c63a-4468-a31c-ef643398528f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:36.706586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd14c6-c63a-4468-a31c-ef643398528f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:36.706861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd14c6-c63a-4468-a31c-ef643398528f None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:36.707525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd14c6-c63a-4468-a31c-ef643398528f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:36.707824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd14c6-c63a-4468-a31c-ef643398528f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:36.709285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd14c6-c63a-4468-a31c-ef643398528f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:36.718878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 583/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:36.724718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b9aca-3c74-4e0a-ae2e-a253d4cb8f2b None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:36.790435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:36.825273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b9aca-3c74-4e0a-ae2e-a253d4cb8f2b None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:36.859361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8901e86-c11c-4b42-a46b-d2f2fb6c248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:36.859993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8901e86-c11c-4b42-a46b-d2f2fb6c248c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:36.860270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8901e86-c11c-4b42-a46b-d2f2fb6c248c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:36.860542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8901e86-c11c-4b42-a46b-d2f2fb6c248c None admin] PATH_INFO: `/v3/users/c6ed86e1f9624e22930369753605ad41` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:36.861678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8901e86-c11c-4b42-a46b-d2f2fb6c248c None admin] RBAC: Authorizing `identity:get_user(user_id=c6ed86e1f9624e22930369753605ad41)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:36.863139 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8901e86-c11c-4b42-a46b-d2f2fb6c248c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:36.864717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 607/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:07:36 2022] GET /identity/v3/users/c6ed86e1f9624e22930369753605ad41 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:36.871039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd14c6-c63a-4468-a31c-ef643398528f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:36.968561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd14c6-c63a-4468-a31c-ef643398528f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:37.001352 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270d9b5-b132-42e1-9120-f5adc170c3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:37.001962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270d9b5-b132-42e1-9120-f5adc170c3e4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:37.002237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270d9b5-b132-42e1-9120-f5adc170c3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:37.002529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270d9b5-b132-42e1-9120-f5adc170c3e4 None admin] PATH_INFO: `/v3/projects/d688d3b369c140378eac8fadaedb5470` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:37.003622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270d9b5-b132-42e1-9120-f5adc170c3e4 None admin] RBAC: Authorizing `identity:get_project(project_id=d688d3b369c140378eac8fadaedb5470)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:37.005054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270d9b5-b132-42e1-9120-f5adc170c3e4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:37.006721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 584/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:07:36 2022] GET /identity/v3/projects/d688d3b369c140378eac8fadaedb5470 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:37.012816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8901e86-c11c-4b42-a46b-d2f2fb6c248c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:37.112713 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8901e86-c11c-4b42-a46b-d2f2fb6c248c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:37.145978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed702c-5700-4246-b787-0bf4a78b3244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:37.146657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed702c-5700-4246-b787-0bf4a78b3244 None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:37.146938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed702c-5700-4246-b787-0bf4a78b3244 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:37.147210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed702c-5700-4246-b787-0bf4a78b3244 None admin] PATH_INFO: `/v3/projects/d688d3b369c140378eac8fadaedb5470/users/c6ed86e1f9624e22930369753605ad41/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:37.149087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed702c-5700-4246-b787-0bf4a78b3244 None admin] RBAC: Authorizing `identity:create_grant(project_id=d688d3b369c140378eac8fadaedb5470, user_id=c6ed86e1f9624e22930369753605ad41, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:37.150554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed702c-5700-4246-b787-0bf4a78b3244 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:37.153321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:37.218997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 608/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:07:37 2022] PUT /identity/v3/projects/d688d3b369c140378eac8fadaedb5470/users/c6ed86e1f9624e22930369753605ad41/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:07:39.128376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a04c9-fbdc-487d-a660-f82494870ffc None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:39.128911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a04c9-fbdc-487d-a660-f82494870ffc None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:39.128911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a04c9-fbdc-487d-a660-f82494870ffc None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:39.130286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 585/1193] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:07:39.135413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e876e-d3f8-4c2a-9491-a2dd1a75deab None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:39.135859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e876e-d3f8-4c2a-9491-a2dd1a75deab None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:39.136021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e876e-d3f8-4c2a-9491-a2dd1a75deab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:39.154228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-840e876e-d3f8-4c2a-9491-a2dd1a75deab None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:07:39.155453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-840e876e-d3f8-4c2a-9491-a2dd1a75deab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:07:39.220948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 609/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:07:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:07:39.492538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a04c9-fbdc-487d-a660-f82494870ffc None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:39.520762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:39.600126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a04c9-fbdc-487d-a660-f82494870ffc None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:39.639931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f09e26-528a-4674-9de2-f32a5de53c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:39.640593 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f09e26-528a-4674-9de2-f32a5de53c10 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:39.640869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f09e26-528a-4674-9de2-f32a5de53c10 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:39.641141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f09e26-528a-4674-9de2-f32a5de53c10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:39.652252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f09e26-528a-4674-9de2-f32a5de53c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-02f09e26-528a-4674-9de2-f32a5de53c10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:07:39.656937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:07:39.658062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:07:39.659140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 586/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:07:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:07:39.664240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e876e-d3f8-4c2a-9491-a2dd1a75deab None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:39.766577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e876e-d3f8-4c2a-9491-a2dd1a75deab None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:39.789911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:39.800788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c39470-ecb5-4c3c-9e64-2773a42d73c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:39.801383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39470-ecb5-4c3c-9e64-2773a42d73c1 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:39.801654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39470-ecb5-4c3c-9e64-2773a42d73c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:39.801918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39470-ecb5-4c3c-9e64-2773a42d73c1 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:39.802606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c39470-ecb5-4c3c-9e64-2773a42d73c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:39.802901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c39470-ecb5-4c3c-9e64-2773a42d73c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:39.804254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c39470-ecb5-4c3c-9e64-2773a42d73c1 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:39.813536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 610/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:07:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:07:39.819361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f09e26-528a-4674-9de2-f32a5de53c10 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:39.905487 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:39.920217 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f09e26-528a-4674-9de2-f32a5de53c10 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:39.954323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e2db0-642d-4549-b9bd-d1e90da59caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:39.955140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e2db0-642d-4549-b9bd-d1e90da59caf None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:39.955422 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e2db0-642d-4549-b9bd-d1e90da59caf None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:39.955694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e2db0-642d-4549-b9bd-d1e90da59caf None admin] PATH_INFO: `/v3/users/c6ed86e1f9624e22930369753605ad41` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:39.956820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e2db0-642d-4549-b9bd-d1e90da59caf None admin] RBAC: Authorizing `identity:get_user(user_id=c6ed86e1f9624e22930369753605ad41)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:39.958244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e2db0-642d-4549-b9bd-d1e90da59caf None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:39.959868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 587/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 22 17:07:39 2022] GET /identity/v3/users/c6ed86e1f9624e22930369753605ad41 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:39.966165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c39470-ecb5-4c3c-9e64-2773a42d73c1 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:40.065177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c39470-ecb5-4c3c-9e64-2773a42d73c1 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:40.097427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd023068-735b-40f5-ba6c-881ead339c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:40.098030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd023068-735b-40f5-ba6c-881ead339c6d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:40.098310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd023068-735b-40f5-ba6c-881ead339c6d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:40.098613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd023068-735b-40f5-ba6c-881ead339c6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:40.151911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:40.164719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd023068-735b-40f5-ba6c-881ead339c6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:40.171440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd023068-735b-40f5-ba6c-881ead339c6d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:40.264583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 611/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:07:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:07:40.270568 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e2db0-642d-4549-b9bd-d1e90da59caf None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:40.373123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e2db0-642d-4549-b9bd-d1e90da59caf None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:40.408281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7338eee-66b1-4245-8511-932b282c4b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:40.408909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7338eee-66b1-4245-8511-932b282c4b83 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:40.409186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7338eee-66b1-4245-8511-932b282c4b83 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:40.409456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7338eee-66b1-4245-8511-932b282c4b83 None admin] PATH_INFO: `/v3/projects/d688d3b369c140378eac8fadaedb5470` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:40.410637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7338eee-66b1-4245-8511-932b282c4b83 None admin] RBAC: Authorizing `identity:get_project(project_id=d688d3b369c140378eac8fadaedb5470)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:40.412080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7338eee-66b1-4245-8511-932b282c4b83 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:40.413738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 588/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 22 17:07:40 2022] GET /identity/v3/projects/d688d3b369c140378eac8fadaedb5470 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:40.420773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd023068-735b-40f5-ba6c-881ead339c6d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:07:40.506371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:40.521552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd023068-735b-40f5-ba6c-881ead339c6d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:07:40.557539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2425d4-e72f-408f-8a9a-85ba98463c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:07:40.558241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2425d4-e72f-408f-8a9a-85ba98463c42 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:07:40.558546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2425d4-e72f-408f-8a9a-85ba98463c42 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:07:40.558822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2425d4-e72f-408f-8a9a-85ba98463c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:07:40.559553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2425d4-e72f-408f-8a9a-85ba98463c42 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=d688d3b369c140378eac8fadaedb5470, user.id=c6ed86e1f9624e22930369753605ad41) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:07:40.559908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2425d4-e72f-408f-8a9a-85ba98463c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:07:40.561350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2425d4-e72f-408f-8a9a-85ba98463c42 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:07:40.567458 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 612/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:07:40 2022] GET /identity/v3/role_assignments?user.id=c6ed86e1f9624e22930369753605ad41&scope.project.id=d688d3b369c140378eac8fadaedb5470&role.id=4b10782d99b74f5bb453b82286012e0c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:07:50.403870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:07:50.564655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=25 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:08:59.535021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc9f67-0434-426a-a1de-faebe6902a75 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:08:59.535653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc9f67-0434-426a-a1de-faebe6902a75 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:08:59.535653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc9f67-0434-426a-a1de-faebe6902a75 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:08:59.537080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 589/1201] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:08:59.542734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9dfa8-f7e7-4aa2-8c35-d4db2b33451e None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:08:59.543167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9dfa8-f7e7-4aa2-8c35-d4db2b33451e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:08:59.543265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9dfa8-f7e7-4aa2-8c35-d4db2b33451e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:08:59.561179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-15f9dfa8-f7e7-4aa2-8c35-d4db2b33451e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:08:59.562365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-15f9dfa8-f7e7-4aa2-8c35-d4db2b33451e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:08:59.632001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 613/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:08:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:08:59.906088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc9f67-0434-426a-a1de-faebe6902a75 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:00.009266 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc9f67-0434-426a-a1de-faebe6902a75 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:00.043221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e823e62-a310-42b3-bfc1-f81825228294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:00.043830 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823e62-a310-42b3-bfc1-f81825228294 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:00.045892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823e62-a310-42b3-bfc1-f81825228294 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:00.045892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823e62-a310-42b3-bfc1-f81825228294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:00.055228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e823e62-a310-42b3-bfc1-f81825228294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:00.056712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e823e62-a310-42b3-bfc1-f81825228294 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-6e823e62-a310-42b3-bfc1-f81825228294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:00.072792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:00.074014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:00.075174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:00.075174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:00.075174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:00.075174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 590/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:00.079978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9dfa8-f7e7-4aa2-8c35-d4db2b33451e None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:00.179027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9dfa8-f7e7-4aa2-8c35-d4db2b33451e None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:00.207589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:00.214112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8d725-4e10-4124-b934-775924c222be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:00.214721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8d725-4e10-4124-b934-775924c222be None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:00.214992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8d725-4e10-4124-b934-775924c222be None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:00.215254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8d725-4e10-4124-b934-775924c222be None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:00.215883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8d725-4e10-4124-b934-775924c222be None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:00.216178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8d725-4e10-4124-b934-775924c222be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:00.217530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8d725-4e10-4124-b934-775924c222be None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:00.228693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 614/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:00.234870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e823e62-a310-42b3-bfc1-f81825228294 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:00.286686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:00.334601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e823e62-a310-42b3-bfc1-f81825228294 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:00.368276 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4a6ed-9503-495f-ae0d-0542e66b296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:00.368918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a6ed-9503-495f-ae0d-0542e66b296a None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:00.369199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a6ed-9503-495f-ae0d-0542e66b296a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:00.369472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a6ed-9503-495f-ae0d-0542e66b296a None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:00.370115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4a6ed-9503-495f-ae0d-0542e66b296a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:00.371606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4a6ed-9503-495f-ae0d-0542e66b296a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:00.562195 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 591/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 22 17:09:00 2022] POST /identity/v3/users => generated 251 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:09:02.525616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622f48-2e34-406f-8447-20c3fc8bef02 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:02.526181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622f48-2e34-406f-8447-20c3fc8bef02 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:02.526181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622f48-2e34-406f-8447-20c3fc8bef02 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:02.527570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 615/1206] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:09:02.533066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93925d-ee4b-46b6-820e-730177d16369 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:02.533535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93925d-ee4b-46b6-820e-730177d16369 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:02.533709 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93925d-ee4b-46b6-820e-730177d16369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:02.553459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-fa93925d-ee4b-46b6-820e-730177d16369 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:09:02.554737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fa93925d-ee4b-46b6-820e-730177d16369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:09:02.626640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 592/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:09:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:09:02.899934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8622f48-2e34-406f-8447-20c3fc8bef02 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:03.003166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8622f48-2e34-406f-8447-20c3fc8bef02 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:03.037146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2749f-11f1-4563-aaa3-d4d2fff8f4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:03.037756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2749f-11f1-4563-aaa3-d4d2fff8f4fd None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:03.038029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2749f-11f1-4563-aaa3-d4d2fff8f4fd None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:03.038292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2749f-11f1-4563-aaa3-d4d2fff8f4fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:03.048810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2749f-11f1-4563-aaa3-d4d2fff8f4fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-53a2749f-11f1-4563-aaa3-d4d2fff8f4fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:09:03.054384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:03.055449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:03.056436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 616/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:09:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:03.061572 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93925d-ee4b-46b6-820e-730177d16369 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:03.103456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:03.166031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93925d-ee4b-46b6-820e-730177d16369 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:03.200040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc24ca-0e36-49df-b972-fc41bedd7c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:03.205523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc24ca-0e36-49df-b972-fc41bedd7c6a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:03.205523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc24ca-0e36-49df-b972-fc41bedd7c6a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:03.205523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc24ca-0e36-49df-b972-fc41bedd7c6a None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:03.205523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc24ca-0e36-49df-b972-fc41bedd7c6a None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:03.205523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc24ca-0e36-49df-b972-fc41bedd7c6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:03.205523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc24ca-0e36-49df-b972-fc41bedd7c6a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:03.212107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 593/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:09:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:03.218060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2749f-11f1-4563-aaa3-d4d2fff8f4fd None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:03.270986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:03.323516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2749f-11f1-4563-aaa3-d4d2fff8f4fd None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:03.358507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d138e8-894d-412c-9f8e-1479b13db4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:03.359170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d138e8-894d-412c-9f8e-1479b13db4f3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:03.359533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d138e8-894d-412c-9f8e-1479b13db4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:03.359777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d138e8-894d-412c-9f8e-1479b13db4f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:03.371277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d138e8-894d-412c-9f8e-1479b13db4f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:03.372771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d138e8-894d-412c-9f8e-1479b13db4f3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-97d138e8-894d-412c-9f8e-1479b13db4f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:03.386772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:03.387921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:03.389015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:03.389015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:03.389015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:03.389015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 617/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:09:03.394061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc24ca-0e36-49df-b972-fc41bedd7c6a None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:03.493745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:03.496547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc24ca-0e36-49df-b972-fc41bedd7c6a None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:03.530303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2844d9-6a8f-4bbd-8ae6-8db1f68e6d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:03.530929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2844d9-6a8f-4bbd-8ae6-8db1f68e6d04 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:03.531207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2844d9-6a8f-4bbd-8ae6-8db1f68e6d04 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:03.531516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2844d9-6a8f-4bbd-8ae6-8db1f68e6d04 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:03.532145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2844d9-6a8f-4bbd-8ae6-8db1f68e6d04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:03.532441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2844d9-6a8f-4bbd-8ae6-8db1f68e6d04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:03.533814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2844d9-6a8f-4bbd-8ae6-8db1f68e6d04 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:03.546245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 594/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:03.552375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d138e8-894d-412c-9f8e-1479b13db4f3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:03.656149 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d138e8-894d-412c-9f8e-1479b13db4f3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:03.657689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:03.690449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897fdc8-87b7-4aee-8057-9f3b8389f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:03.691104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897fdc8-87b7-4aee-8057-9f3b8389f1aa None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:03.691381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897fdc8-87b7-4aee-8057-9f3b8389f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:03.691651 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897fdc8-87b7-4aee-8057-9f3b8389f1aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:03.699057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897fdc8-87b7-4aee-8057-9f3b8389f1aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:03.700494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897fdc8-87b7-4aee-8057-9f3b8389f1aa None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-5897fdc8-87b7-4aee-8057-9f3b8389f1aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:09:03.712120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:03.713314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:03.714569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 618/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:03.719280 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2844d9-6a8f-4bbd-8ae6-8db1f68e6d04 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:03.820449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2844d9-6a8f-4bbd-8ae6-8db1f68e6d04 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:03.854909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a27de-fb81-4260-900e-85939f2a4120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:03.855513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a27de-fb81-4260-900e-85939f2a4120 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:03.855792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a27de-fb81-4260-900e-85939f2a4120 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:03.856063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a27de-fb81-4260-900e-85939f2a4120 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:03.856782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a27de-fb81-4260-900e-85939f2a4120 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:03.857088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a27de-fb81-4260-900e-85939f2a4120 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:03.858581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a27de-fb81-4260-900e-85939f2a4120 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:03.875499 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 595/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 22 17:09:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:03.882047 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897fdc8-87b7-4aee-8057-9f3b8389f1aa None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:03.984418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897fdc8-87b7-4aee-8057-9f3b8389f1aa None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:04.017991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bbb54-990d-40e6-a641-d80cb4c7e1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:04.018616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bbb54-990d-40e6-a641-d80cb4c7e1b5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:04.018895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bbb54-990d-40e6-a641-d80cb4c7e1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:04.019198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bbb54-990d-40e6-a641-d80cb4c7e1b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:04.050131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:04.087994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bbb54-990d-40e6-a641-d80cb4c7e1b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:04.089367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bbb54-990d-40e6-a641-d80cb4c7e1b5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:04.188297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 619/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:09:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:09:04.194893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a27de-fb81-4260-900e-85939f2a4120 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:04.214373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:04.293338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a27de-fb81-4260-900e-85939f2a4120 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:04.329844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c1991-43c8-4618-9b72-2e8290564e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:04.330475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c1991-43c8-4618-9b72-2e8290564e8c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:04.330781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c1991-43c8-4618-9b72-2e8290564e8c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:04.331052 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c1991-43c8-4618-9b72-2e8290564e8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:04.341500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c1991-43c8-4618-9b72-2e8290564e8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:04.342918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c1991-43c8-4618-9b72-2e8290564e8c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-fd7c1991-43c8-4618-9b72-2e8290564e8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:04.357085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:04.364663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:04.365756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:04.365756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:04.365756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:04.365756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 596/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:04.366064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bbb54-990d-40e6-a641-d80cb4c7e1b5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:04.426134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:04.464886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bbb54-990d-40e6-a641-d80cb4c7e1b5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:04.499459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175cf83-9886-4247-b2aa-3a4448fc90cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:04.500079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175cf83-9886-4247-b2aa-3a4448fc90cb None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:04.500361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175cf83-9886-4247-b2aa-3a4448fc90cb None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:04.500632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175cf83-9886-4247-b2aa-3a4448fc90cb None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:04.501274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175cf83-9886-4247-b2aa-3a4448fc90cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:04.501574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175cf83-9886-4247-b2aa-3a4448fc90cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:04.502974 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175cf83-9886-4247-b2aa-3a4448fc90cb None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:04.515252 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 620/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:04.521121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c1991-43c8-4618-9b72-2e8290564e8c None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:04.593548 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:04.625019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c1991-43c8-4618-9b72-2e8290564e8c None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:04.657732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafb14a-9eba-4cae-960a-1d579ac0bc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:04.658426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafb14a-9eba-4cae-960a-1d579ac0bc74 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:04.658759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafb14a-9eba-4cae-960a-1d579ac0bc74 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:04.659030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafb14a-9eba-4cae-960a-1d579ac0bc74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:04.666178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafb14a-9eba-4cae-960a-1d579ac0bc74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:04.667928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafb14a-9eba-4cae-960a-1d579ac0bc74 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-7fafb14a-9eba-4cae-960a-1d579ac0bc74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:09:04.680057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:04.682822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:04.683906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:09:04.683906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:09:04.683906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:09:04.683906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:09:04.683906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:09:04.683906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:09:04.683906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:04.683906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:04.683906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 597/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:09:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:04.687166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175cf83-9886-4247-b2aa-3a4448fc90cb None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:04.790566 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175cf83-9886-4247-b2aa-3a4448fc90cb None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:04.813229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:04.825314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618583e-a196-4790-bc48-a265b52646c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:04.825926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618583e-a196-4790-bc48-a265b52646c0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:04.826205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618583e-a196-4790-bc48-a265b52646c0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:04.826497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618583e-a196-4790-bc48-a265b52646c0 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:04.827205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618583e-a196-4790-bc48-a265b52646c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:04.827510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618583e-a196-4790-bc48-a265b52646c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:04.829000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618583e-a196-4790-bc48-a265b52646c0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:04.841161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 621/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:09:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:04.847433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafb14a-9eba-4cae-960a-1d579ac0bc74 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:04.949945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafb14a-9eba-4cae-960a-1d579ac0bc74 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:04.979066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:04.984940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee2be9-80cd-4a7d-82ec-4c576d6bca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:04.985549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee2be9-80cd-4a7d-82ec-4c576d6bca8e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:04.985821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee2be9-80cd-4a7d-82ec-4c576d6bca8e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:04.986644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee2be9-80cd-4a7d-82ec-4c576d6bca8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:04.986945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee2be9-80cd-4a7d-82ec-4c576d6bca8e None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=26b5021285c44db9b05f9d15ceaf797f) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:04.987243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee2be9-80cd-4a7d-82ec-4c576d6bca8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:04.988657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee2be9-80cd-4a7d-82ec-4c576d6bca8e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:04.994614 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 598/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:09:04 2022] GET /identity/v3/role_assignments?user.id=26b5021285c44db9b05f9d15ceaf797f&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:06.906170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d7215-5b36-4f62-8d6c-8f313c3e1174 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:06.906739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d7215-5b36-4f62-8d6c-8f313c3e1174 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:06.906739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d7215-5b36-4f62-8d6c-8f313c3e1174 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:06.908106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 622/1220] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:09:06.914323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0d1d9-1c6a-454e-94a6-6ea8209a0dd7 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:06.914797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0d1d9-1c6a-454e-94a6-6ea8209a0dd7 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:06.914899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0d1d9-1c6a-454e-94a6-6ea8209a0dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:06.934002 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-14e0d1d9-1c6a-454e-94a6-6ea8209a0dd7 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:09:06.935239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-14e0d1d9-1c6a-454e-94a6-6ea8209a0dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:09:07.001795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 599/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:09:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:09:07.274542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d7215-5b36-4f62-8d6c-8f313c3e1174 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:07.380138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d7215-5b36-4f62-8d6c-8f313c3e1174 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:07.414826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bc6c5-f748-4def-82a2-f086850aae74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:07.415446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bc6c5-f748-4def-82a2-f086850aae74 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:07.415721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bc6c5-f748-4def-82a2-f086850aae74 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:07.415989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bc6c5-f748-4def-82a2-f086850aae74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:07.427113 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bc6c5-f748-4def-82a2-f086850aae74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-6b2bc6c5-f748-4def-82a2-f086850aae74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:09:07.431979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:07.433164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:07.434164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 623/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:09:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:07.439248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0d1d9-1c6a-454e-94a6-6ea8209a0dd7 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:07.539627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0d1d9-1c6a-454e-94a6-6ea8209a0dd7 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:07.562822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:07.573798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6472e7f-2920-4261-8514-fc68bc0fb417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:07.574405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472e7f-2920-4261-8514-fc68bc0fb417 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:07.574836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472e7f-2920-4261-8514-fc68bc0fb417 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:07.575106 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472e7f-2920-4261-8514-fc68bc0fb417 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:07.575764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6472e7f-2920-4261-8514-fc68bc0fb417 None admin] RBAC: Adding query filter params (name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:07.576064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6472e7f-2920-4261-8514-fc68bc0fb417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:07.577430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6472e7f-2920-4261-8514-fc68bc0fb417 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:07.586152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 600/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:09:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:07.591992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bc6c5-f748-4def-82a2-f086850aae74 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:07.637856 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:07.693301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bc6c5-f748-4def-82a2-f086850aae74 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:07.728759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182db61-4b63-40de-b73c-53fcb0e45936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:07.729454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182db61-4b63-40de-b73c-53fcb0e45936 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:07.729772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182db61-4b63-40de-b73c-53fcb0e45936 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:07.730084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182db61-4b63-40de-b73c-53fcb0e45936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:07.741209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182db61-4b63-40de-b73c-53fcb0e45936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:07.742800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182db61-4b63-40de-b73c-53fcb0e45936 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-6182db61-4b63-40de-b73c-53fcb0e45936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:07.757434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:07.762228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:07.763378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:07.763378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:07.763378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:07.763378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 624/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:07.764462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6472e7f-2920-4261-8514-fc68bc0fb417 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:07.864882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6472e7f-2920-4261-8514-fc68bc0fb417 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:07.898937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffd444-a8e1-44c7-a46e-0fad91dcb1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:07.899531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffd444-a8e1-44c7-a46e-0fad91dcb1b9 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:07.899808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffd444-a8e1-44c7-a46e-0fad91dcb1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:07.900074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffd444-a8e1-44c7-a46e-0fad91dcb1b9 None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:07.900706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffd444-a8e1-44c7-a46e-0fad91dcb1b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:07.901005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffd444-a8e1-44c7-a46e-0fad91dcb1b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:07.902367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffd444-a8e1-44c7-a46e-0fad91dcb1b9 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:07.915063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 601/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:09:07.921267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182db61-4b63-40de-b73c-53fcb0e45936 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:08.015321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:08.022503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182db61-4b63-40de-b73c-53fcb0e45936 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:08.055762 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f850b4e-b151-48a6-b00a-0d864e4ec656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:08.056481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f850b4e-b151-48a6-b00a-0d864e4ec656 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:08.056832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f850b4e-b151-48a6-b00a-0d864e4ec656 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:08.057172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f850b4e-b151-48a6-b00a-0d864e4ec656 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:08.064909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f850b4e-b151-48a6-b00a-0d864e4ec656 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:08.066381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f850b4e-b151-48a6-b00a-0d864e4ec656 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-2f850b4e-b151-48a6-b00a-0d864e4ec656 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:09:08.077468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:08.078653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:08.079822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 625/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:08.085156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffd444-a8e1-44c7-a46e-0fad91dcb1b9 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:08.133963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:08.187813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffd444-a8e1-44c7-a46e-0fad91dcb1b9 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:08.221768 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea8247-e5b1-4fb2-ab08-18e275f14ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:08.222448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea8247-e5b1-4fb2-ab08-18e275f14ee6 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:08.222792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea8247-e5b1-4fb2-ab08-18e275f14ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:08.223062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea8247-e5b1-4fb2-ab08-18e275f14ee6 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:08.224341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea8247-e5b1-4fb2-ab08-18e275f14ee6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:08.224642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea8247-e5b1-4fb2-ab08-18e275f14ee6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:08.226110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea8247-e5b1-4fb2-ab08-18e275f14ee6 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:08.243122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 602/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 22 17:09:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:08.248935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f850b4e-b151-48a6-b00a-0d864e4ec656 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:08.353351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f850b4e-b151-48a6-b00a-0d864e4ec656 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:08.387144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b06d0c-8075-421d-aa74-37edeb8484d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:08.387698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b06d0c-8075-421d-aa74-37edeb8484d3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:08.387975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b06d0c-8075-421d-aa74-37edeb8484d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:08.388245 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b06d0c-8075-421d-aa74-37edeb8484d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:08.399744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b06d0c-8075-421d-aa74-37edeb8484d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:08.401212 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b06d0c-8075-421d-aa74-37edeb8484d3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-86b06d0c-8075-421d-aa74-37edeb8484d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:08.415473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:08.416653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:08.418286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:08.418286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:08.418286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:08.418286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 626/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:08.422934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea8247-e5b1-4fb2-ab08-18e275f14ee6 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:08.523837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea8247-e5b1-4fb2-ab08-18e275f14ee6 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:08.526158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:08.557724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e8123-3eff-4537-ab0a-828b37dfd41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:08.558319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e8123-3eff-4537-ab0a-828b37dfd41e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:08.558626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e8123-3eff-4537-ab0a-828b37dfd41e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:08.558903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e8123-3eff-4537-ab0a-828b37dfd41e None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:08.559536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e8123-3eff-4537-ab0a-828b37dfd41e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:08.559836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e8123-3eff-4537-ab0a-828b37dfd41e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:08.561221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e8123-3eff-4537-ab0a-828b37dfd41e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:08.573377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 603/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:08.579293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b06d0c-8075-421d-aa74-37edeb8484d3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:08.589614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:08.683999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b06d0c-8075-421d-aa74-37edeb8484d3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:08.717809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd87fc9-65c3-4c0f-8cd1-bfd1bac0164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:08.718433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd87fc9-65c3-4c0f-8cd1-bfd1bac0164d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:08.718744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd87fc9-65c3-4c0f-8cd1-bfd1bac0164d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:08.719012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd87fc9-65c3-4c0f-8cd1-bfd1bac0164d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:08.725739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd87fc9-65c3-4c0f-8cd1-bfd1bac0164d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:08.727341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd87fc9-65c3-4c0f-8cd1-bfd1bac0164d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-bdd87fc9-65c3-4c0f-8cd1-bfd1bac0164d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:09:08.738279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:08.740097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:08.741003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:09:08.741003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:09:08.741003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:09:08.741003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:09:08.741003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:09:08.741003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:09:08.741003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:08.741003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:08.741003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 627/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:09:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:08.746175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e8123-3eff-4537-ab0a-828b37dfd41e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:08.847911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e8123-3eff-4537-ab0a-828b37dfd41e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:08.882216 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695bd4c-e835-4276-937d-3e2ad55de11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:08.882853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695bd4c-e835-4276-937d-3e2ad55de11f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:08.883134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695bd4c-e835-4276-937d-3e2ad55de11f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:08.883404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695bd4c-e835-4276-937d-3e2ad55de11f None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:08.884084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695bd4c-e835-4276-937d-3e2ad55de11f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:08.884388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695bd4c-e835-4276-937d-3e2ad55de11f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:08.885809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695bd4c-e835-4276-937d-3e2ad55de11f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:08.897644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 604/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:09:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:08.904844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd87fc9-65c3-4c0f-8cd1-bfd1bac0164d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:08.971091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:09.006763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd87fc9-65c3-4c0f-8cd1-bfd1bac0164d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:09.039979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7f4dd-d13b-4488-a9ae-f1f76e5dae87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:09.040712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7f4dd-d13b-4488-a9ae-f1f76e5dae87 None admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:09.040989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7f4dd-d13b-4488-a9ae-f1f76e5dae87 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:09.041258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7f4dd-d13b-4488-a9ae-f1f76e5dae87 None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/26b5021285c44db9b05f9d15ceaf797f/roles/14a34ac79fe64a978eca3e5f9089e72f` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:09.057036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7f4dd-d13b-4488-a9ae-f1f76e5dae87 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=26b5021285c44db9b05f9d15ceaf797f, role_id=14a34ac79fe64a978eca3e5f9089e72f)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:09.058573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7f4dd-d13b-4488-a9ae-f1f76e5dae87 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:09.181534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 628/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:09:08 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/26b5021285c44db9b05f9d15ceaf797f/roles/14a34ac79fe64a978eca3e5f9089e72f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:09:11.105715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5417bce-cc7b-4e07-9398-661db869683d None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:11.106274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5417bce-cc7b-4e07-9398-661db869683d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:11.106274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5417bce-cc7b-4e07-9398-661db869683d None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:11.107661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 605/1233] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:09:11.113070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc57901-ce49-467b-b862-ada2260acf81 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:11.113488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc57901-ce49-467b-b862-ada2260acf81 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:11.113649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc57901-ce49-467b-b862-ada2260acf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:11.132345 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6dc57901-ce49-467b-b862-ada2260acf81 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:09:11.133512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6dc57901-ce49-467b-b862-ada2260acf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:09:11.201360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 629/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:09:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:09:11.476177 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5417bce-cc7b-4e07-9398-661db869683d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:11.501754 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:11.580651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5417bce-cc7b-4e07-9398-661db869683d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:11.614999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c8b66-c7fd-4546-bc84-53a91a7d9366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:11.615622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c8b66-c7fd-4546-bc84-53a91a7d9366 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:11.615900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c8b66-c7fd-4546-bc84-53a91a7d9366 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:11.616167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c8b66-c7fd-4546-bc84-53a91a7d9366 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:11.627434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c8b66-c7fd-4546-bc84-53a91a7d9366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-d39c8b66-c7fd-4546-bc84-53a91a7d9366 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:09:11.633019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:11.638757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:11.639924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 606/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 22 17:09:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:11.641994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc57901-ce49-467b-b862-ada2260acf81 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:11.688083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:11.741208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc57901-ce49-467b-b862-ada2260acf81 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:11.775741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf6e42-f7af-4f50-afa9-dc271caa8293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:11.776421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf6e42-f7af-4f50-afa9-dc271caa8293 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:11.776617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf6e42-f7af-4f50-afa9-dc271caa8293 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:11.776884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf6e42-f7af-4f50-afa9-dc271caa8293 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:11.777605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf6e42-f7af-4f50-afa9-dc271caa8293 None admin] RBAC: Adding query filter params (name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:11.777902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf6e42-f7af-4f50-afa9-dc271caa8293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:11.779291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf6e42-f7af-4f50-afa9-dc271caa8293 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:11.787635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 630/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 22 17:09:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:11.793691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c8b66-c7fd-4546-bc84-53a91a7d9366 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:11.889489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:11.905482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c8b66-c7fd-4546-bc84-53a91a7d9366 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:11.941205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520bc84-7e26-4baa-973f-933074392ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:11.941837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520bc84-7e26-4baa-973f-933074392ace None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:11.942712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520bc84-7e26-4baa-973f-933074392ace None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:11.943205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520bc84-7e26-4baa-973f-933074392ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:11.954821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520bc84-7e26-4baa-973f-933074392ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:11.956624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520bc84-7e26-4baa-973f-933074392ace None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-4520bc84-7e26-4baa-973f-933074392ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:11.974120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:11.975316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:11.976375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:11.976375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:11.976375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:11.976375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 607/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:11.981420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf6e42-f7af-4f50-afa9-dc271caa8293 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:12.085186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf6e42-f7af-4f50-afa9-dc271caa8293 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:12.094393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:12.120349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758dd371-5050-4096-a15c-2a85f53cd40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:12.121017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dd371-5050-4096-a15c-2a85f53cd40c None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:12.121251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dd371-5050-4096-a15c-2a85f53cd40c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:12.121561 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dd371-5050-4096-a15c-2a85f53cd40c None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:12.122205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758dd371-5050-4096-a15c-2a85f53cd40c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:12.122533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758dd371-5050-4096-a15c-2a85f53cd40c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:12.123897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758dd371-5050-4096-a15c-2a85f53cd40c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:12.135747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 631/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:12.141530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520bc84-7e26-4baa-973f-933074392ace None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:12.242751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520bc84-7e26-4baa-973f-933074392ace None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:12.277660 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a61677-8fab-4d66-b7b9-8d3c8c5680ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:12.278279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61677-8fab-4d66-b7b9-8d3c8c5680ec None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:12.278584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61677-8fab-4d66-b7b9-8d3c8c5680ec None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:12.278857 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61677-8fab-4d66-b7b9-8d3c8c5680ec None admin] PATH_INFO: `/v3/users/placement` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:12.281615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:12.287279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a61677-8fab-4d66-b7b9-8d3c8c5680ec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:12.288793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a61677-8fab-4d66-b7b9-8d3c8c5680ec None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-44a61677-8fab-4d66-b7b9-8d3c8c5680ec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:09:12.300379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:12.302093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:12.303433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 608/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:12.308157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758dd371-5050-4096-a15c-2a85f53cd40c None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:12.413203 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758dd371-5050-4096-a15c-2a85f53cd40c None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:12.447633 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f42aa7-baea-4b09-8233-6f228affcdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:12.448233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f42aa7-baea-4b09-8233-6f228affcdd7 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:12.448505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f42aa7-baea-4b09-8233-6f228affcdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:12.448770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f42aa7-baea-4b09-8233-6f228affcdd7 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:12.449477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f42aa7-baea-4b09-8233-6f228affcdd7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:12.449774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f42aa7-baea-4b09-8233-6f228affcdd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:12.451208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f42aa7-baea-4b09-8233-6f228affcdd7 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:12.468758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 632/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 22 17:09:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:12.475710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a61677-8fab-4d66-b7b9-8d3c8c5680ec None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:12.579615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a61677-8fab-4d66-b7b9-8d3c8c5680ec None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:12.615601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8a9e7-8ff3-4f17-87ce-0a7672803387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:12.620305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8a9e7-8ff3-4f17-87ce-0a7672803387 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:12.620305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8a9e7-8ff3-4f17-87ce-0a7672803387 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:12.620305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8a9e7-8ff3-4f17-87ce-0a7672803387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:12.679715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:12.686205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8a9e7-8ff3-4f17-87ce-0a7672803387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:12.687702 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8a9e7-8ff3-4f17-87ce-0a7672803387 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:12.785018 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 609/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:09:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:09:12.791806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f42aa7-baea-4b09-8233-6f228affcdd7 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:12.819166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:12.891152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f42aa7-baea-4b09-8233-6f228affcdd7 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:12.925109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3bc8-2bfc-40d1-8c2f-721b154c15e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:12.925725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3bc8-2bfc-40d1-8c2f-721b154c15e0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:12.925997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3bc8-2bfc-40d1-8c2f-721b154c15e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:12.926325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3bc8-2bfc-40d1-8c2f-721b154c15e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:12.943486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd3bc8-2bfc-40d1-8c2f-721b154c15e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:12.943486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd3bc8-2bfc-40d1-8c2f-721b154c15e0 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-8cdd3bc8-2bfc-40d1-8c2f-721b154c15e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:12.954107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:12.955291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:12.956233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:12.956233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:12.956233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:12.956233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 633/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:12.961720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8a9e7-8ff3-4f17-87ce-0a7672803387 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:13.056650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:13.062435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8a9e7-8ff3-4f17-87ce-0a7672803387 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:13.097816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a37095-202f-4d08-8eaa-7d41b5e03f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:13.098590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a37095-202f-4d08-8eaa-7d41b5e03f7e None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:13.098886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a37095-202f-4d08-8eaa-7d41b5e03f7e None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:13.099152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a37095-202f-4d08-8eaa-7d41b5e03f7e None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:13.099794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a37095-202f-4d08-8eaa-7d41b5e03f7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:13.100098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a37095-202f-4d08-8eaa-7d41b5e03f7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:13.101487 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a37095-202f-4d08-8eaa-7d41b5e03f7e None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:13.113206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 610/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:13.119027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3bc8-2bfc-40d1-8c2f-721b154c15e0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:13.200194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:13.224552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3bc8-2bfc-40d1-8c2f-721b154c15e0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:13.259840 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd82402-3d3c-4f68-8bcc-0469406d8d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:13.260485 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd82402-3d3c-4f68-8bcc-0469406d8d12 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:13.260765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd82402-3d3c-4f68-8bcc-0469406d8d12 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:13.261061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd82402-3d3c-4f68-8bcc-0469406d8d12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:13.268431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd82402-3d3c-4f68-8bcc-0469406d8d12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:13.269884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd82402-3d3c-4f68-8bcc-0469406d8d12 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ddd82402-3d3c-4f68-8bcc-0469406d8d12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:09:13.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:13.282372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:13.283457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:09:13.283457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:09:13.283457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:09:13.283457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:09:13.283457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:09:13.283457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:09:13.283457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:13.283457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:13.283457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 634/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:13.288437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a37095-202f-4d08-8eaa-7d41b5e03f7e None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:13.394184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a37095-202f-4d08-8eaa-7d41b5e03f7e None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:13.433532 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240c66d-fb28-4501-a26b-8767ca76793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:13.434277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240c66d-fb28-4501-a26b-8767ca76793a None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:13.434792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240c66d-fb28-4501-a26b-8767ca76793a None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:13.435085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240c66d-fb28-4501-a26b-8767ca76793a None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:13.435851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240c66d-fb28-4501-a26b-8767ca76793a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:13.436154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240c66d-fb28-4501-a26b-8767ca76793a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:13.437892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240c66d-fb28-4501-a26b-8767ca76793a None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:13.452285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 611/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:09:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:13.458588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd82402-3d3c-4f68-8bcc-0469406d8d12 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:13.562334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd82402-3d3c-4f68-8bcc-0469406d8d12 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:13.597970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5cbfe-e4be-4851-803c-d0271ad203db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:13.599151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5cbfe-e4be-4851-803c-d0271ad203db None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:13.599422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5cbfe-e4be-4851-803c-d0271ad203db None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:13.599687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5cbfe-e4be-4851-803c-d0271ad203db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:13.600406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5cbfe-e4be-4851-803c-d0271ad203db None admin] RBAC: Adding query filter params (role.id=14a34ac79fe64a978eca3e5f9089e72f, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=26b5021285c44db9b05f9d15ceaf797f) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:13.600709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5cbfe-e4be-4851-803c-d0271ad203db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:13.602127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5cbfe-e4be-4851-803c-d0271ad203db None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:13.607868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 635/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:09:13 2022] GET /identity/v3/role_assignments?user.id=26b5021285c44db9b05f9d15ceaf797f&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=14a34ac79fe64a978eca3e5f9089e72f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:15.584551 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956500-b307-437a-955a-a32133b66194 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:15.585109 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956500-b307-437a-955a-a32133b66194 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:15.585109 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956500-b307-437a-955a-a32133b66194 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:15.586434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 612/1247] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:09:15.591797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223238b-b519-4cce-ad9b-bbf2cadd27cb None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:15.592231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223238b-b519-4cce-ad9b-bbf2cadd27cb None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:15.592444 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223238b-b519-4cce-ad9b-bbf2cadd27cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:15.599289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:15.611489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8223238b-b519-4cce-ad9b-bbf2cadd27cb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:09:15.612659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8223238b-b519-4cce-ad9b-bbf2cadd27cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:09:15.682653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 636/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:09:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:09:15.946496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5956500-b307-437a-955a-a32133b66194 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:15.964699 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:16.047308 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5956500-b307-437a-955a-a32133b66194 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:16.080525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1dd915-fef3-4cc2-8bda-be6cd963ee83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:16.081141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1dd915-fef3-4cc2-8bda-be6cd963ee83 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:16.081412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1dd915-fef3-4cc2-8bda-be6cd963ee83 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:16.081676 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1dd915-fef3-4cc2-8bda-be6cd963ee83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:16.091595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1dd915-fef3-4cc2-8bda-be6cd963ee83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-af1dd915-fef3-4cc2-8bda-be6cd963ee83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:09:16.096285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:16.097437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:16.099448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 613/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:09:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:16.104026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223238b-b519-4cce-ad9b-bbf2cadd27cb None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:16.202532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223238b-b519-4cce-ad9b-bbf2cadd27cb None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:16.233350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:16.238363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e96d27-677e-49f1-859f-db1ce8f36359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:16.239233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e96d27-677e-49f1-859f-db1ce8f36359 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:16.239533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e96d27-677e-49f1-859f-db1ce8f36359 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:16.239809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e96d27-677e-49f1-859f-db1ce8f36359 None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:16.240574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e96d27-677e-49f1-859f-db1ce8f36359 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:16.240903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e96d27-677e-49f1-859f-db1ce8f36359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:16.242365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e96d27-677e-49f1-859f-db1ce8f36359 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:16.251107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 637/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:09:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:16.256978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1dd915-fef3-4cc2-8bda-be6cd963ee83 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:16.339155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:16.362074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1dd915-fef3-4cc2-8bda-be6cd963ee83 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:16.395637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4ed4-937e-48b5-948e-d40a9972bae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:16.405257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4ed4-937e-48b5-948e-d40a9972bae1 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:16.405257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4ed4-937e-48b5-948e-d40a9972bae1 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:16.405257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4ed4-937e-48b5-948e-d40a9972bae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:16.406415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c4ed4-937e-48b5-948e-d40a9972bae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:16.407896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c4ed4-937e-48b5-948e-d40a9972bae1 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-a29c4ed4-937e-48b5-948e-d40a9972bae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:16.422589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:16.423866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:16.424791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:16.424791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:16.424791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:16.424791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 614/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 17:09:16.429636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e96d27-677e-49f1-859f-db1ce8f36359 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:16.529979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e96d27-677e-49f1-859f-db1ce8f36359 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:16.564906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff37d06-c450-46b5-8ac1-5a8bcd888643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:16.573547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff37d06-c450-46b5-8ac1-5a8bcd888643 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:16.573547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff37d06-c450-46b5-8ac1-5a8bcd888643 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:16.573547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff37d06-c450-46b5-8ac1-5a8bcd888643 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:16.573547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff37d06-c450-46b5-8ac1-5a8bcd888643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:16.573547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff37d06-c450-46b5-8ac1-5a8bcd888643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:16.573547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff37d06-c450-46b5-8ac1-5a8bcd888643 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:16.581037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 638/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:16.587369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4ed4-937e-48b5-948e-d40a9972bae1 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:16.691623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4ed4-937e-48b5-948e-d40a9972bae1 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:16.718085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:16.726228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4035a25-5d27-43aa-98d1-7d2757fd21ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:16.726902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4035a25-5d27-43aa-98d1-7d2757fd21ac None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:16.727182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4035a25-5d27-43aa-98d1-7d2757fd21ac None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:16.727454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4035a25-5d27-43aa-98d1-7d2757fd21ac None admin] PATH_INFO: `/v3/users/placement` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:16.734740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4035a25-5d27-43aa-98d1-7d2757fd21ac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:16.736206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4035a25-5d27-43aa-98d1-7d2757fd21ac None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-e4035a25-5d27-43aa-98d1-7d2757fd21ac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:09:16.747526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:16.749342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:16.750581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 615/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:16.755957 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff37d06-c450-46b5-8ac1-5a8bcd888643 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:16.813586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:16.858977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff37d06-c450-46b5-8ac1-5a8bcd888643 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:16.893641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b61b5-50ea-419f-8a2a-c704ce02aaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:16.894246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b61b5-50ea-419f-8a2a-c704ce02aaef None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:16.894548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b61b5-50ea-419f-8a2a-c704ce02aaef None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:16.894818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b61b5-50ea-419f-8a2a-c704ce02aaef None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:16.896148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b61b5-50ea-419f-8a2a-c704ce02aaef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:16.896441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b61b5-50ea-419f-8a2a-c704ce02aaef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:16.897862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b61b5-50ea-419f-8a2a-c704ce02aaef None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:16.914960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 639/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 22 17:09:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:16.920829 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4035a25-5d27-43aa-98d1-7d2757fd21ac None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:17.023663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4035a25-5d27-43aa-98d1-7d2757fd21ac None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:17.057418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7136290-8e3e-493d-9867-41f71e3caab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:17.058006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7136290-8e3e-493d-9867-41f71e3caab4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:17.058275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7136290-8e3e-493d-9867-41f71e3caab4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:17.058577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7136290-8e3e-493d-9867-41f71e3caab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:17.112980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:17.125781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7136290-8e3e-493d-9867-41f71e3caab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:17.127307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7136290-8e3e-493d-9867-41f71e3caab4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:17.224960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 616/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:09:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:09:17.230839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b61b5-50ea-419f-8a2a-c704ce02aaef None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:17.332745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b61b5-50ea-419f-8a2a-c704ce02aaef None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:17.346175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:17.368902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541010b0-cdb8-4e08-8b80-cbc6052412d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:17.369458 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541010b0-cdb8-4e08-8b80-cbc6052412d5 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:17.369740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541010b0-cdb8-4e08-8b80-cbc6052412d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:17.370014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541010b0-cdb8-4e08-8b80-cbc6052412d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:17.379982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541010b0-cdb8-4e08-8b80-cbc6052412d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:17.381423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541010b0-cdb8-4e08-8b80-cbc6052412d5 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-541010b0-cdb8-4e08-8b80-cbc6052412d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:17.395575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:17.396542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:17.397490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:17.397490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:17.397490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:17.397490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 640/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:17.402464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7136290-8e3e-493d-9867-41f71e3caab4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:17.491756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:17.503812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7136290-8e3e-493d-9867-41f71e3caab4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:17.537834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ad3e0-e460-407c-8bba-b247c82fe27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:17.538430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad3e0-e460-407c-8bba-b247c82fe27b None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:17.538741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad3e0-e460-407c-8bba-b247c82fe27b None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:17.539010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad3e0-e460-407c-8bba-b247c82fe27b None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:17.539731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ad3e0-e460-407c-8bba-b247c82fe27b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:17.540037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ad3e0-e460-407c-8bba-b247c82fe27b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:17.541400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ad3e0-e460-407c-8bba-b247c82fe27b None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:17.553444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 617/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:17.559146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541010b0-cdb8-4e08-8b80-cbc6052412d5 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:17.662603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541010b0-cdb8-4e08-8b80-cbc6052412d5 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:17.698103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69713d28-a7cd-497e-b159-d14eea524ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:17.698767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69713d28-a7cd-497e-b159-d14eea524ff3 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:17.699102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69713d28-a7cd-497e-b159-d14eea524ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:17.699398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69713d28-a7cd-497e-b159-d14eea524ff3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:17.706208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69713d28-a7cd-497e-b159-d14eea524ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:17.709036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69713d28-a7cd-497e-b159-d14eea524ff3 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-69713d28-a7cd-497e-b159-d14eea524ff3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:09:17.719299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:17.720360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:17.721326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:09:17.721326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:09:17.721326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:09:17.721326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:09:17.721326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:09:17.721326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:09:17.721326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:17.721326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:17.721326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 641/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:09:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:17.727226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ad3e0-e460-407c-8bba-b247c82fe27b None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:17.826329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ad3e0-e460-407c-8bba-b247c82fe27b None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:17.861777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ac1c0-b0e1-4202-b8e7-9b155d07b81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:17.862381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ac1c0-b0e1-4202-b8e7-9b155d07b81c None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:17.862695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ac1c0-b0e1-4202-b8e7-9b155d07b81c None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:17.862969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ac1c0-b0e1-4202-b8e7-9b155d07b81c None admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:17.863845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ac1c0-b0e1-4202-b8e7-9b155d07b81c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:17.864160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ac1c0-b0e1-4202-b8e7-9b155d07b81c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:17.865591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ac1c0-b0e1-4202-b8e7-9b155d07b81c None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:17.880731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 618/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:09:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:17.884428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69713d28-a7cd-497e-b159-d14eea524ff3 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:17.912146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:17.987851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69713d28-a7cd-497e-b159-d14eea524ff3 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:18.020692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f4675-44a4-42eb-8aa6-45566d410d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:18.021577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f4675-44a4-42eb-8aa6-45566d410d69 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:18.021856 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f4675-44a4-42eb-8aa6-45566d410d69 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:18.022126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f4675-44a4-42eb-8aa6-45566d410d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:18.022867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f4675-44a4-42eb-8aa6-45566d410d69 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=26b5021285c44db9b05f9d15ceaf797f) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:18.023175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f4675-44a4-42eb-8aa6-45566d410d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:18.024582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f4675-44a4-42eb-8aa6-45566d410d69 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:18.029968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 642/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:09:17 2022] GET /identity/v3/role_assignments?user.id=26b5021285c44db9b05f9d15ceaf797f&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=4b10782d99b74f5bb453b82286012e0c => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:19.939363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50ed77-2a60-423e-b92e-17589ff59f25 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:19.939921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50ed77-2a60-423e-b92e-17589ff59f25 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:19.939921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50ed77-2a60-423e-b92e-17589ff59f25 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:19.941261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 619/1261] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:09:19.946794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a43fd-aefd-478d-ba72-731ec41cc3dd None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:19.947235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a43fd-aefd-478d-ba72-731ec41cc3dd None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:19.951650 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a43fd-aefd-478d-ba72-731ec41cc3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:19.966449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-335a43fd-aefd-478d-ba72-731ec41cc3dd None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:09:19.968985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-335a43fd-aefd-478d-ba72-731ec41cc3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:09:20.035167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 643/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:09:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:09:20.309290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50ed77-2a60-423e-b92e-17589ff59f25 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:20.320362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:20.413457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50ed77-2a60-423e-b92e-17589ff59f25 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:20.450065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32e339-562e-4330-9bee-0c3b07460344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:20.450708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32e339-562e-4330-9bee-0c3b07460344 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:20.450985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32e339-562e-4330-9bee-0c3b07460344 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:20.451253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32e339-562e-4330-9bee-0c3b07460344 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:20.462463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32e339-562e-4330-9bee-0c3b07460344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-7d32e339-562e-4330-9bee-0c3b07460344 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:09:20.467375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:20.468519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:20.469626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 620/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:09:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:20.474658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a43fd-aefd-478d-ba72-731ec41cc3dd None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:20.493454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:20.576800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a43fd-aefd-478d-ba72-731ec41cc3dd None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:20.610950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06144e-d005-4d4f-926b-08791ac3767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:20.611551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06144e-d005-4d4f-926b-08791ac3767e None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:20.611825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06144e-d005-4d4f-926b-08791ac3767e None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:20.612094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06144e-d005-4d4f-926b-08791ac3767e None admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:20.612746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06144e-d005-4d4f-926b-08791ac3767e None admin] RBAC: Adding query filter params (name=admin) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:20.613046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06144e-d005-4d4f-926b-08791ac3767e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:20.614429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06144e-d005-4d4f-926b-08791ac3767e None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:20.622745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 644/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:09:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:20.628768 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32e339-562e-4330-9bee-0c3b07460344 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:20.696115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:20.730193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32e339-562e-4330-9bee-0c3b07460344 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:20.764837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8fed5-ef8c-4ac9-b824-a423987c60a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:20.765540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8fed5-ef8c-4ac9-b824-a423987c60a0 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:20.765817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8fed5-ef8c-4ac9-b824-a423987c60a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:20.766086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8fed5-ef8c-4ac9-b824-a423987c60a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:20.777938 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8fed5-ef8c-4ac9-b824-a423987c60a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:20.779469 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8fed5-ef8c-4ac9-b824-a423987c60a0 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-c0e8fed5-ef8c-4ac9-b824-a423987c60a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:20.794407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:20.795649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:20.796646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:20.796646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:20.796646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:20.796646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 621/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:20.802358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06144e-d005-4d4f-926b-08791ac3767e None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:20.886702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:20.905934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06144e-d005-4d4f-926b-08791ac3767e None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:20.940003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821776ce-bea1-4776-8168-ab1f7ae01940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:20.940609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821776ce-bea1-4776-8168-ab1f7ae01940 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:20.940885 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821776ce-bea1-4776-8168-ab1f7ae01940 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:20.941154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821776ce-bea1-4776-8168-ab1f7ae01940 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:20.941815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821776ce-bea1-4776-8168-ab1f7ae01940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:20.942901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821776ce-bea1-4776-8168-ab1f7ae01940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:20.944294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821776ce-bea1-4776-8168-ab1f7ae01940 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:20.956127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 645/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:20.962340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8fed5-ef8c-4ac9-b824-a423987c60a0 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:21.064319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8fed5-ef8c-4ac9-b824-a423987c60a0 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:21.085109 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:21.099864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5597595-94c4-4c78-89c8-5721428e60de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:21.100488 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5597595-94c4-4c78-89c8-5721428e60de None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:21.100763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5597595-94c4-4c78-89c8-5721428e60de None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:21.101030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5597595-94c4-4c78-89c8-5721428e60de None admin] PATH_INFO: `/v3/users/placement` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:21.108432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5597595-94c4-4c78-89c8-5721428e60de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:21.110047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5597595-94c4-4c78-89c8-5721428e60de None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-e5597595-94c4-4c78-89c8-5721428e60de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:09:21.121564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:21.122787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:21.123790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 622/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:21.129122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821776ce-bea1-4776-8168-ab1f7ae01940 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:21.231796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821776ce-bea1-4776-8168-ab1f7ae01940 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:21.266281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15a9e5-e3c2-436b-9aa2-5da6de82c061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:21.266918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15a9e5-e3c2-436b-9aa2-5da6de82c061 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:21.267199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15a9e5-e3c2-436b-9aa2-5da6de82c061 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:21.267471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15a9e5-e3c2-436b-9aa2-5da6de82c061 None admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:21.268198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15a9e5-e3c2-436b-9aa2-5da6de82c061 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:21.268503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15a9e5-e3c2-436b-9aa2-5da6de82c061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:21.270048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15a9e5-e3c2-436b-9aa2-5da6de82c061 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:21.279064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:21.288294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 646/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 22 17:09:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:09:21.294674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5597595-94c4-4c78-89c8-5721428e60de None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:21.396220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5597595-94c4-4c78-89c8-5721428e60de None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:21.430986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c844f12-6bea-4982-a4f7-5ba6c02d91ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:21.431648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c844f12-6bea-4982-a4f7-5ba6c02d91ab None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:21.431924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c844f12-6bea-4982-a4f7-5ba6c02d91ab None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:21.432188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c844f12-6bea-4982-a4f7-5ba6c02d91ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:21.443193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c844f12-6bea-4982-a4f7-5ba6c02d91ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:21.444665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c844f12-6bea-4982-a4f7-5ba6c02d91ab None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-9c844f12-6bea-4982-a4f7-5ba6c02d91ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:21.459612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:21.460762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:21.461791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:21.461791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:21.461791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:21.461791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 623/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:21.466941 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15a9e5-e3c2-436b-9aa2-5da6de82c061 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:21.572653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15a9e5-e3c2-436b-9aa2-5da6de82c061 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:21.607928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737f808-e658-4c05-88ce-88b72739a767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:21.608534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737f808-e658-4c05-88ce-88b72739a767 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:21.620597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737f808-e658-4c05-88ce-88b72739a767 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:21.620597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737f808-e658-4c05-88ce-88b72739a767 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:21.620597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737f808-e658-4c05-88ce-88b72739a767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:21.620597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737f808-e658-4c05-88ce-88b72739a767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:21.620597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737f808-e658-4c05-88ce-88b72739a767 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:21.622874 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 647/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:21.628770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c844f12-6bea-4982-a4f7-5ba6c02d91ab None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:21.662968 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:21.735710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c844f12-6bea-4982-a4f7-5ba6c02d91ab None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:21.771524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b07a2-40d8-47f2-a5ff-11e2399addd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:21.772149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b07a2-40d8-47f2-a5ff-11e2399addd2 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:21.772428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b07a2-40d8-47f2-a5ff-11e2399addd2 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:21.772699 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b07a2-40d8-47f2-a5ff-11e2399addd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:21.779536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b07a2-40d8-47f2-a5ff-11e2399addd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:21.781008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b07a2-40d8-47f2-a5ff-11e2399addd2 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-542b07a2-40d8-47f2-a5ff-11e2399addd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:09:21.793145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:21.794160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:21.795122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:09:21.795122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:09:21.795122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:09:21.795122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:09:21.795122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:09:21.795122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:09:21.795122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:21.795122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:21.795122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 624/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:21.801061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737f808-e658-4c05-88ce-88b72739a767 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:21.870107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:21.911934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737f808-e658-4c05-88ce-88b72739a767 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:21.945327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217031d-170b-46cb-85f0-c25607ccfe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:21.946266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217031d-170b-46cb-85f0-c25607ccfe11 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:21.946573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217031d-170b-46cb-85f0-c25607ccfe11 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:21.946845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217031d-170b-46cb-85f0-c25607ccfe11 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:21.947536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217031d-170b-46cb-85f0-c25607ccfe11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:21.947836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217031d-170b-46cb-85f0-c25607ccfe11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:21.949293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217031d-170b-46cb-85f0-c25607ccfe11 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:21.962569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 648/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:21.969073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b07a2-40d8-47f2-a5ff-11e2399addd2 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:22.062940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:22.072240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b07a2-40d8-47f2-a5ff-11e2399addd2 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:22.107047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8ad32-b747-4ec7-baa6-5b2f931afe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:22.107733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ad32-b747-4ec7-baa6-5b2f931afe31 None admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:22.108003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ad32-b747-4ec7-baa6-5b2f931afe31 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:22.108269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ad32-b747-4ec7-baa6-5b2f931afe31 None admin] PATH_INFO: `/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/26b5021285c44db9b05f9d15ceaf797f/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:22.110136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8ad32-b747-4ec7-baa6-5b2f931afe31 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a99c33cfab44e22b69495fc410fa077, user_id=26b5021285c44db9b05f9d15ceaf797f, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:22.111583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8ad32-b747-4ec7-baa6-5b2f931afe31 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:22.247190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 625/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 22 17:09:21 2022] PUT /identity/v3/projects/9a99c33cfab44e22b69495fc410fa077/users/26b5021285c44db9b05f9d15ceaf797f/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:09:24.172046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11461d63-d15c-4064-8de8-f0aaa8e3fa0f None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:24.172584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11461d63-d15c-4064-8de8-f0aaa8e3fa0f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:24.172672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11461d63-d15c-4064-8de8-f0aaa8e3fa0f None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:24.173976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 649/1274] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:09:24.179562 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7fdcc-48dc-430c-92c0-bd4db06a709a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:24.179961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7fdcc-48dc-430c-92c0-bd4db06a709a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:24.180111 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7fdcc-48dc-430c-92c0-bd4db06a709a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:24.199786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-67b7fdcc-48dc-430c-92c0-bd4db06a709a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:09:24.200969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-67b7fdcc-48dc-430c-92c0-bd4db06a709a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:09:24.281407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 626/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:09:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:09:24.555005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11461d63-d15c-4064-8de8-f0aaa8e3fa0f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:24.658680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11461d63-d15c-4064-8de8-f0aaa8e3fa0f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:24.676610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:24.694932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da507e9-39d6-45af-9d21-0563a1972be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:24.695553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da507e9-39d6-45af-9d21-0563a1972be0 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:24.695833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da507e9-39d6-45af-9d21-0563a1972be0 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:24.696103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da507e9-39d6-45af-9d21-0563a1972be0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:24.706728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da507e9-39d6-45af-9d21-0563a1972be0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-8da507e9-39d6-45af-9d21-0563a1972be0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 17:09:24.711501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:24.712626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:24.713594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 650/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 22 17:09:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:24.718732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7fdcc-48dc-430c-92c0-bd4db06a709a None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:24.798726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:24.818653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7fdcc-48dc-430c-92c0-bd4db06a709a None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:24.853300 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b21d6-0ca4-4ce4-8b68-cd5327d4fd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:24.853996 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b21d6-0ca4-4ce4-8b68-cd5327d4fd08 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:24.854556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b21d6-0ca4-4ce4-8b68-cd5327d4fd08 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:24.854914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b21d6-0ca4-4ce4-8b68-cd5327d4fd08 None admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:24.856038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b21d6-0ca4-4ce4-8b68-cd5327d4fd08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:24.856429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b21d6-0ca4-4ce4-8b68-cd5327d4fd08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:24.857840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b21d6-0ca4-4ce4-8b68-cd5327d4fd08 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:24.867314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 627/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 22 17:09:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:24.873154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da507e9-39d6-45af-9d21-0563a1972be0 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:24.976993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da507e9-39d6-45af-9d21-0563a1972be0 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:25.010747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf5419-2341-4d8b-8a20-c3480076a3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:25.011369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf5419-2341-4d8b-8a20-c3480076a3ae None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:25.011634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf5419-2341-4d8b-8a20-c3480076a3ae None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:25.011901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf5419-2341-4d8b-8a20-c3480076a3ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:25.021681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf5419-2341-4d8b-8a20-c3480076a3ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:25.023147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf5419-2341-4d8b-8a20-c3480076a3ae None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-7cbf5419-2341-4d8b-8a20-c3480076a3ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:25.037692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:25.041686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:25.043187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:25.043187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:25.043187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:25.043187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 651/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:25.045088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b21d6-0ca4-4ce4-8b68-cd5327d4fd08 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:25.146947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b21d6-0ca4-4ce4-8b68-cd5327d4fd08 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:25.182947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcbbe4-549a-4852-b39e-ce22fff7f67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:25.183672 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcbbe4-549a-4852-b39e-ce22fff7f67f None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:25.183953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcbbe4-549a-4852-b39e-ce22fff7f67f None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:25.184229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcbbe4-549a-4852-b39e-ce22fff7f67f None admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:25.184868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcbbe4-549a-4852-b39e-ce22fff7f67f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:25.185171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcbbe4-549a-4852-b39e-ce22fff7f67f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:25.186569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcbbe4-549a-4852-b39e-ce22fff7f67f None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:25.198663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 628/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:25.206365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf5419-2341-4d8b-8a20-c3480076a3ae None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:25.236635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:25.307773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf5419-2341-4d8b-8a20-c3480076a3ae None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:25.343749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cd3cd-e0bf-442c-810d-32092c2d239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:25.344376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd3cd-e0bf-442c-810d-32092c2d239d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:25.344647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd3cd-e0bf-442c-810d-32092c2d239d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:25.344916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd3cd-e0bf-442c-810d-32092c2d239d None admin] PATH_INFO: `/v3/users/placement` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:25.352923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cd3cd-e0bf-442c-810d-32092c2d239d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:25.354400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cd3cd-e0bf-442c-810d-32092c2d239d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-a12cd3cd-e0bf-442c-810d-32092c2d239d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 17:09:25.365887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:25.366968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:25.368059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 652/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:25.372925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcbbe4-549a-4852-b39e-ce22fff7f67f None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:25.376898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:25.476153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcbbe4-549a-4852-b39e-ce22fff7f67f None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:25.510603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141a5e7-3e22-4c61-abfc-2f9521b47ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:25.511204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141a5e7-3e22-4c61-abfc-2f9521b47ef5 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:25.511482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141a5e7-3e22-4c61-abfc-2f9521b47ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:25.511752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141a5e7-3e22-4c61-abfc-2f9521b47ef5 None admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:25.512515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141a5e7-3e22-4c61-abfc-2f9521b47ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:25.512850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141a5e7-3e22-4c61-abfc-2f9521b47ef5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:25.514342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141a5e7-3e22-4c61-abfc-2f9521b47ef5 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:25.532073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 629/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 22 17:09:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:25.539457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cd3cd-e0bf-442c-810d-32092c2d239d None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:25.631954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:25.641322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cd3cd-e0bf-442c-810d-32092c2d239d None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:25.675643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716fb05-f425-425c-80ad-a0799bce3f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:25.676270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716fb05-f425-425c-80ad-a0799bce3f95 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:25.676538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716fb05-f425-425c-80ad-a0799bce3f95 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:25.676801 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716fb05-f425-425c-80ad-a0799bce3f95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:25.745412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716fb05-f425-425c-80ad-a0799bce3f95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:25.746825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716fb05-f425-425c-80ad-a0799bce3f95 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:25.835803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:25.845816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 653/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 22 17:09:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 17:09:25.852168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141a5e7-3e22-4c61-abfc-2f9521b47ef5 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:25.914844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:25.953429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141a5e7-3e22-4c61-abfc-2f9521b47ef5 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:25.987295 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e68dc-8d6a-4851-a1e4-7b4191af0e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:25.987911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e68dc-8d6a-4851-a1e4-7b4191af0e77 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:25.988179 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e68dc-8d6a-4851-a1e4-7b4191af0e77 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:25.988440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e68dc-8d6a-4851-a1e4-7b4191af0e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:26.000310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e68dc-8d6a-4851-a1e4-7b4191af0e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:26.001748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e68dc-8d6a-4851-a1e4-7b4191af0e77 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-eb0e68dc-8d6a-4851-a1e4-7b4191af0e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 17:09:26.016979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:26.018191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 17:09:26.019505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 17:09:26.019505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 17:09:26.019505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:26.019505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 630/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:09:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:26.024168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716fb05-f425-425c-80ad-a0799bce3f95 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:26.121393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716fb05-f425-425c-80ad-a0799bce3f95 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:26.154899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750960cc-fbc7-4cce-806a-62712d546480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:26.155491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750960cc-fbc7-4cce-806a-62712d546480 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:26.155769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750960cc-fbc7-4cce-806a-62712d546480 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:26.156033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750960cc-fbc7-4cce-806a-62712d546480 None admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:26.156662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750960cc-fbc7-4cce-806a-62712d546480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:26.156953 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750960cc-fbc7-4cce-806a-62712d546480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:26.158296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750960cc-fbc7-4cce-806a-62712d546480 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:26.169753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 654/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 22 17:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:09:26.175498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e68dc-8d6a-4851-a1e4-7b4191af0e77 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:26.281290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e68dc-8d6a-4851-a1e4-7b4191af0e77 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:26.295228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:26.315373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791310b-6395-4946-bff4-651520cde2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:26.315998 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791310b-6395-4946-bff4-651520cde2c4 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:26.316268 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791310b-6395-4946-bff4-651520cde2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:26.316538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791310b-6395-4946-bff4-651520cde2c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:26.324285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791310b-6395-4946-bff4-651520cde2c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:26.325717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791310b-6395-4946-bff4-651520cde2c4 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application [None req-6791310b-6395-4946-bff4-651520cde2c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 17:09:26.337261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:26.338387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:26.339504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 17:09:26.339504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 17:09:26.339504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 17:09:26.339504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 17:09:26.339504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 17:09:26.339504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 17:09:26.339504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 17:09:26.339504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: ERROR keystone.server.flask.application Aug 22 17:09:26.339504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 631/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 22 17:09:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:26.344765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750960cc-fbc7-4cce-806a-62712d546480 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:26.395912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:26.445451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750960cc-fbc7-4cce-806a-62712d546480 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:26.478006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936af74-1f46-4904-9734-b59d901d1657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:26.478635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936af74-1f46-4904-9734-b59d901d1657 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:26.478913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936af74-1f46-4904-9734-b59d901d1657 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:26.479181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936af74-1f46-4904-9734-b59d901d1657 None admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:26.479864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936af74-1f46-4904-9734-b59d901d1657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:26.480166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936af74-1f46-4904-9734-b59d901d1657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:26.481601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936af74-1f46-4904-9734-b59d901d1657 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:26.496914 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 655/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 22 17:09:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:26.502883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791310b-6395-4946-bff4-651520cde2c4 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:26.605436 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791310b-6395-4946-bff4-651520cde2c4 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:26.641152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f49f2-b2fe-4427-8cc9-da4b6aad1207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:26.641758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f49f2-b2fe-4427-8cc9-da4b6aad1207 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:26.642035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f49f2-b2fe-4427-8cc9-da4b6aad1207 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:26.642303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f49f2-b2fe-4427-8cc9-da4b6aad1207 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:26.643071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f49f2-b2fe-4427-8cc9-da4b6aad1207 None admin] RBAC: Adding query filter params (role.id=4b10782d99b74f5bb453b82286012e0c, scope.project.id=9a99c33cfab44e22b69495fc410fa077, user.id=26b5021285c44db9b05f9d15ceaf797f) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:26.643382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f49f2-b2fe-4427-8cc9-da4b6aad1207 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:26.644994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f49f2-b2fe-4427-8cc9-da4b6aad1207 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:26.650860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 632/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 22 17:09:26 2022] GET /identity/v3/role_assignments?user.id=26b5021285c44db9b05f9d15ceaf797f&scope.project.id=9a99c33cfab44e22b69495fc410fa077&role.id=4b10782d99b74f5bb453b82286012e0c => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:28.649893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd852fab-5296-4822-9464-2620d6249c4d None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:28.650447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd852fab-5296-4822-9464-2620d6249c4d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:28.650447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd852fab-5296-4822-9464-2620d6249c4d None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:28.651846 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 656/1288] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:09:28.657442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe87b0-c7fb-4f58-aa2c-a3b0f3202a48 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:28.657888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe87b0-c7fb-4f58-aa2c-a3b0f3202a48 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:28.658002 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe87b0-c7fb-4f58-aa2c-a3b0f3202a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:28.677339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-5fbe87b0-c7fb-4f58-aa2c-a3b0f3202a48 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:09:28.678574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5fbe87b0-c7fb-4f58-aa2c-a3b0f3202a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:09:28.713776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:28.751773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 633/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:09:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:09:29.026415 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd852fab-5296-4822-9464-2620d6249c4d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:29.131709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd852fab-5296-4822-9464-2620d6249c4d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:29.142763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:29.166040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b3a16-205d-4cda-be2b-911d4900759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:29.166701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b3a16-205d-4cda-be2b-911d4900759f None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:29.167227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b3a16-205d-4cda-be2b-911d4900759f None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:29.167491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b3a16-205d-4cda-be2b-911d4900759f None admin] PATH_INFO: `/v3/services/placement` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:29.168064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b3a16-205d-4cda-be2b-911d4900759f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:29.169419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b3a16-205d-4cda-be2b-911d4900759f None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-019b3a16-205d-4cda-be2b-911d4900759f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:29.179392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:29.180700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:29.181582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 657/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 22 17:09:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:29.186445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe87b0-c7fb-4f58-aa2c-a3b0f3202a48 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:29.288874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe87b0-c7fb-4f58-aa2c-a3b0f3202a48 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:29.324845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a178af4-9d37-4069-9f9f-b1c94e86ef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:29.325570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a178af4-9d37-4069-9f9f-b1c94e86ef2d None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:29.326039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a178af4-9d37-4069-9f9f-b1c94e86ef2d None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:29.326452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a178af4-9d37-4069-9f9f-b1c94e86ef2d None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:29.327216 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a178af4-9d37-4069-9f9f-b1c94e86ef2d None admin] RBAC: Adding query filter params (name=placement) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:29.327636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a178af4-9d37-4069-9f9f-b1c94e86ef2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:29.329205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a178af4-9d37-4069-9f9f-b1c94e86ef2d None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:29.334766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 634/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 22 17:09:29 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:29.340876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b3a16-205d-4cda-be2b-911d4900759f None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:29.443228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b3a16-205d-4cda-be2b-911d4900759f None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:29.478837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73d3a8-08de-4732-bd18-0e3a61e95c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:29.479443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73d3a8-08de-4732-bd18-0e3a61e95c2d None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:29.479720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73d3a8-08de-4732-bd18-0e3a61e95c2d None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:29.479987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73d3a8-08de-4732-bd18-0e3a61e95c2d None admin] PATH_INFO: `/v3/services` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:29.480616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73d3a8-08de-4732-bd18-0e3a61e95c2d None admin] RBAC: Adding query filter params (type=placement) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:29.480981 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73d3a8-08de-4732-bd18-0e3a61e95c2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:29.482381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73d3a8-08de-4732-bd18-0e3a61e95c2d None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:29.486990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 658/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 22 17:09:29 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:09:31.427767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ec171-c610-4d6b-9823-00a9c00706fe None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:31.428333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ec171-c610-4d6b-9823-00a9c00706fe None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:31.428333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ec171-c610-4d6b-9823-00a9c00706fe None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:31.429703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 635/1293] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:09:31.435090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16c52d-f9ed-452f-b27a-aaf284784c79 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:31.435565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16c52d-f9ed-452f-b27a-aaf284784c79 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:31.435692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16c52d-f9ed-452f-b27a-aaf284784c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:31.454216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-3b16c52d-f9ed-452f-b27a-aaf284784c79 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:09:31.455494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3b16c52d-f9ed-452f-b27a-aaf284784c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:09:31.487231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:31.525907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 659/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:09:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 17:09:31.805973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ec171-c610-4d6b-9823-00a9c00706fe None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:31.850673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:31.928137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ec171-c610-4d6b-9823-00a9c00706fe None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:31.962925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdf67b-581d-47db-8502-0057103d2662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:31.963521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdf67b-581d-47db-8502-0057103d2662 None admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:31.963794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdf67b-581d-47db-8502-0057103d2662 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:31.964060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdf67b-581d-47db-8502-0057103d2662 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:31.964699 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdf67b-581d-47db-8502-0057103d2662 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:31.966046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdf67b-581d-47db-8502-0057103d2662 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:32.054932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 636/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 22 17:09:31 2022] POST /identity/v3/services => generated 246 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:09:34.026276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3866a9-be60-4467-9921-f2289783c896 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:34.026938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3866a9-be60-4467-9921-f2289783c896 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:34.026938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3866a9-be60-4467-9921-f2289783c896 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:34.028246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 660/1296] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:09:34.033496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b15223-de11-4944-9346-23f5d76afe8e None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:34.033911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b15223-de11-4944-9346-23f5d76afe8e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:34.034039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b15223-de11-4944-9346-23f5d76afe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:34.053850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-67b15223-de11-4944-9346-23f5d76afe8e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:09:34.055044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-67b15223-de11-4944-9346-23f5d76afe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:09:34.127555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 637/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:09:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:09:34.401155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3866a9-be60-4467-9921-f2289783c896 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:34.504523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3866a9-be60-4467-9921-f2289783c896 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:34.539397 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72b8e5-4691-46e0-843b-6d48f8438fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:34.540021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72b8e5-4691-46e0-843b-6d48f8438fd9 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:34.540307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72b8e5-4691-46e0-843b-6d48f8438fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:34.540567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72b8e5-4691-46e0-843b-6d48f8438fd9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:34.541148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72b8e5-4691-46e0-843b-6d48f8438fd9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:34.542891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72b8e5-4691-46e0-843b-6d48f8438fd9 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-ed72b8e5-4691-46e0-843b-6d48f8438fd9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:34.552472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:34.553643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:34.554837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 661/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 22 17:09:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:34.559723 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b15223-de11-4944-9346-23f5d76afe8e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:34.578952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:34.659584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b15223-de11-4944-9346-23f5d76afe8e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:34.694355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22ecc0-427f-445d-9b64-ead396c8ff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:34.694984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22ecc0-427f-445d-9b64-ead396c8ff84 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:34.695259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22ecc0-427f-445d-9b64-ead396c8ff84 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:34.695528 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22ecc0-427f-445d-9b64-ead396c8ff84 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:34.696153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22ecc0-427f-445d-9b64-ead396c8ff84 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:34.696451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22ecc0-427f-445d-9b64-ead396c8ff84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:34.697859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22ecc0-427f-445d-9b64-ead396c8ff84 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:34.703020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 638/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 22 17:09:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:34.709757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72b8e5-4691-46e0-843b-6d48f8438fd9 None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:34.740452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:34.810868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72b8e5-4691-46e0-843b-6d48f8438fd9 None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:34.846644 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ad2f6-c85f-4c62-b4d9-c83d7a1fe227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:34.847320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ad2f6-c85f-4c62-b4d9-c83d7a1fe227 None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:34.847601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ad2f6-c85f-4c62-b4d9-c83d7a1fe227 None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:34.847876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ad2f6-c85f-4c62-b4d9-c83d7a1fe227 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:34.848600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ad2f6-c85f-4c62-b4d9-c83d7a1fe227 None admin] RBAC: Adding query filter params (interface=public, service_id=80a98754733749908b1accf4dc85464e, region_id=RegionOne) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:34.848907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ad2f6-c85f-4c62-b4d9-c83d7a1fe227 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:34.850285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ad2f6-c85f-4c62-b4d9-c83d7a1fe227 None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:34.855361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 662/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 22 17:09:34 2022] GET /identity/v3/endpoints?service_id=80a98754733749908b1accf4dc85464e&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:34.862145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22ecc0-427f-445d-9b64-ead396c8ff84 None admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:34.944567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:34.962564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22ecc0-427f-445d-9b64-ead396c8ff84 None admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:34.996603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf10281-a722-437c-b28f-12abcecd62b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:34.997195 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf10281-a722-437c-b28f-12abcecd62b3 None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:34.997466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf10281-a722-437c-b28f-12abcecd62b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:34.997728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf10281-a722-437c-b28f-12abcecd62b3 None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:34.998317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf10281-a722-437c-b28f-12abcecd62b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:34.999700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf10281-a722-437c-b28f-12abcecd62b3 None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:35.007459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 639/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 22 17:09:34 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:09:36.962013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28da145-089a-460f-a2ed-d2a2806c5ebf None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:36.962593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28da145-089a-460f-a2ed-d2a2806c5ebf None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:36.962593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28da145-089a-460f-a2ed-d2a2806c5ebf None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:36.963940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 663/1302] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:09:36.969249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13895665-8a5e-4482-a3dc-fbaaca53cf86 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:36.969663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13895665-8a5e-4482-a3dc-fbaaca53cf86 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:36.969796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13895665-8a5e-4482-a3dc-fbaaca53cf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:36.989163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-13895665-8a5e-4482-a3dc-fbaaca53cf86 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:09:36.990371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-13895665-8a5e-4482-a3dc-fbaaca53cf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:09:37.060930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 640/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:09:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 17:09:37.344765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28da145-089a-460f-a2ed-d2a2806c5ebf None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:37.441983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:37.448891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28da145-089a-460f-a2ed-d2a2806c5ebf None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:37.483261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d93be-0588-44af-8bdc-a579b168e26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:37.483884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d93be-0588-44af-8bdc-a579b168e26a None admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:37.484160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d93be-0588-44af-8bdc-a579b168e26a None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:37.484426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d93be-0588-44af-8bdc-a579b168e26a None admin] PATH_INFO: `/v3/services/placement` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:37.484994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d93be-0588-44af-8bdc-a579b168e26a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:37.486409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d93be-0588-44af-8bdc-a579b168e26a None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application [None req-d19d93be-0588-44af-8bdc-a579b168e26a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 17:09:37.495510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 17:09:37.496741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application with Lock( Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self._enter() Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application return self.creator() Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application created_value = creator( Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: ERROR keystone.server.flask.application Aug 22 17:09:37.497710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 664/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 22 17:09:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 17:09:37.502773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13895665-8a5e-4482-a3dc-fbaaca53cf86 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:37.580338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:37.606628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13895665-8a5e-4482-a3dc-fbaaca53cf86 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:37.643748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5571666-40a6-42ea-b86f-7146eecdbcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:37.647643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5571666-40a6-42ea-b86f-7146eecdbcde None admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:37.647643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5571666-40a6-42ea-b86f-7146eecdbcde None admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:37.647643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5571666-40a6-42ea-b86f-7146eecdbcde None admin] PATH_INFO: `/v3/services` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:37.647643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5571666-40a6-42ea-b86f-7146eecdbcde None admin] RBAC: Adding query filter params (name=placement) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:09:37.647643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5571666-40a6-42ea-b86f-7146eecdbcde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:37.648255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5571666-40a6-42ea-b86f-7146eecdbcde None admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:37.653779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 641/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 22 17:09:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:09:37.659719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d93be-0588-44af-8bdc-a579b168e26a None admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:09:37.770622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d93be-0588-44af-8bdc-a579b168e26a None admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:09:37.807561 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb54761-b585-4469-937f-26530d01a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:09:37.808186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb54761-b585-4469-937f-26530d01a32c None admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:09:37.808462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb54761-b585-4469-937f-26530d01a32c None admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:09:37.808732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb54761-b585-4469-937f-26530d01a32c None admin] PATH_INFO: `/v3/endpoints` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:09:37.809406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb54761-b585-4469-937f-26530d01a32c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:09:37.810839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb54761-b585-4469-937f-26530d01a32c None admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:09:37.904511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 665/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:09:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:09:47.650542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=30 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:09:47.899672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:15.980219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc7dec-62d6-428e-9fea-a5497322f920 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:15.981071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc7dec-62d6-428e-9fea-a5497322f920 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:15.981215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc7dec-62d6-428e-9fea-a5497322f920 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:15.993462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7f1d8-e2c9-4f29-852c-9079d832920d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:15.994308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7f1d8-e2c9-4f29-852c-9079d832920d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:15.994452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7f1d8-e2c9-4f29-852c-9079d832920d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:16.000331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 642/1307] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:15 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:16.180928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-5bc7f1d8-e2c9-4f29-852c-9079d832920d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:16.182979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5bc7f1d8-e2c9-4f29-852c-9079d832920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:16.480239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 666/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:17.085989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5584f-d693-4c04-a63b-de3356509875 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:17.088149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5584f-d693-4c04-a63b-de3356509875 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:17.089194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5584f-d693-4c04-a63b-de3356509875 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:17.093196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 643/1309] 10.222.0.20 () {54 vars in 965 bytes} [Mon Aug 22 17:17:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:17.104165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f5503-6b1c-4a31-b5c6-c5cc1b148633 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:17.104921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f5503-6b1c-4a31-b5c6-c5cc1b148633 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:17.105320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f5503-6b1c-4a31-b5c6-c5cc1b148633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:17.274637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c97f5503-6b1c-4a31-b5c6-c5cc1b148633 None None] MFA Rules not processed for user `c29a91e099ba4cd5886fd910d978fb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:17.277056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c97f5503-6b1c-4a31-b5c6-c5cc1b148633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:17.323313 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:17.459138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 667/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Mon Aug 22 17:17:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:17:17.474969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5584f-d693-4c04-a63b-de3356509875 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:17.478780 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-eca5584f-d693-4c04-a63b-de3356509875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:17.693978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5584f-d693-4c04-a63b-de3356509875 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:17.822925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8089b23-fa7d-4fbe-bd39-53120a00eff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:17.823713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8089b23-fa7d-4fbe-bd39-53120a00eff5 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:17.824354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8089b23-fa7d-4fbe-bd39-53120a00eff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:17.824653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8089b23-fa7d-4fbe-bd39-53120a00eff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:17.826689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f8089b23-fa7d-4fbe-bd39-53120a00eff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:17.921273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:18.073964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8089b23-fa7d-4fbe-bd39-53120a00eff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:18.075730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8089b23-fa7d-4fbe-bd39-53120a00eff5 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:18.116283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 644/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:17:20.776481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b592988-0d61-41da-af3e-fccb416a3c7d None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:20.777045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b592988-0d61-41da-af3e-fccb416a3c7d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:20.777045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b592988-0d61-41da-af3e-fccb416a3c7d None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:20.778666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 668/1312] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:20.785644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d51042-1cf1-4279-a82a-c30ef328f514 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:20.786238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d51042-1cf1-4279-a82a-c30ef328f514 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:20.786238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d51042-1cf1-4279-a82a-c30ef328f514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:20.871500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-f7d51042-1cf1-4279-a82a-c30ef328f514 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:20.872753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f7d51042-1cf1-4279-a82a-c30ef328f514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:20.999241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 645/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:17:21.517609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fbb9c-458b-48b2-8d6c-4e1458e27b34 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:21.517888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fbb9c-458b-48b2-8d6c-4e1458e27b34 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:21.518163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fbb9c-458b-48b2-8d6c-4e1458e27b34 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:21.519615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 669/1314] 10.222.0.20 () {54 vars in 965 bytes} [Mon Aug 22 17:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:21.528982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6eed84-fc89-42e6-b1d0-c50a4057e2d4 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:21.529247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6eed84-fc89-42e6-b1d0-c50a4057e2d4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:21.529513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6eed84-fc89-42e6-b1d0-c50a4057e2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:21.544644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:21.627231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ee6eed84-fc89-42e6-b1d0-c50a4057e2d4 None None] MFA Rules not processed for user `c29a91e099ba4cd5886fd910d978fb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:21.628550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ee6eed84-fc89-42e6-b1d0-c50a4057e2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:21.721629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 646/1315] 10.222.0.20 () {58 vars in 1062 bytes} [Mon Aug 22 17:17:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:21.731671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fbb9c-458b-48b2-8d6c-4e1458e27b34 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:21.733440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e87fbb9c-458b-48b2-8d6c-4e1458e27b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:21.862797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fbb9c-458b-48b2-8d6c-4e1458e27b34 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:21.949982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:21.954108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb47b69-f89a-4fb7-8b8e-9c301e33d39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:21.954781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb47b69-f89a-4fb7-8b8e-9c301e33d39d service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:21.955061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb47b69-f89a-4fb7-8b8e-9c301e33d39d service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:21.955340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb47b69-f89a-4fb7-8b8e-9c301e33d39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:21.957173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fbb47b69-f89a-4fb7-8b8e-9c301e33d39d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:22.091197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb47b69-f89a-4fb7-8b8e-9c301e33d39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:22.092764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb47b69-f89a-4fb7-8b8e-9c301e33d39d service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:22.115413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 670/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:17:24.986016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb55181-3576-4f39-be97-8ce7be03e3c3 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:24.986646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb55181-3576-4f39-be97-8ce7be03e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:24.986646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb55181-3576-4f39-be97-8ce7be03e3c3 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:24.988159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 647/1317] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:24.994643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11bf51-02d0-436b-ad14-a70f3238bdd3 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:24.995457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11bf51-02d0-436b-ad14-a70f3238bdd3 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:24.995457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11bf51-02d0-436b-ad14-a70f3238bdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:25.078315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-9d11bf51-02d0-436b-ad14-a70f3238bdd3 None None] MFA Rules not processed for user `734f29275e5547fcb788acba34124924`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:25.079714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9d11bf51-02d0-436b-ad14-a70f3238bdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:25.147461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:25.201579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 671/1318] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:25.541896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb55181-3576-4f39-be97-8ce7be03e3c3 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:25.541896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9cb55181-3576-4f39-be97-8ce7be03e3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:25.898145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb55181-3576-4f39-be97-8ce7be03e3c3 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:25.898145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:25.898145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a99b7-1733-4ab2-8d87-3a64b832d84c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:25.898145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a99b7-1733-4ab2-8d87-3a64b832d84c service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:25.899304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a99b7-1733-4ab2-8d87-3a64b832d84c service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:25.899304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a99b7-1733-4ab2-8d87-3a64b832d84c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:25.899304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c13a99b7-1733-4ab2-8d87-3a64b832d84c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:25.899304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a99b7-1733-4ab2-8d87-3a64b832d84c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:25.899304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a99b7-1733-4ab2-8d87-3a64b832d84c service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:25.899304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 648/1319] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:17:28.991118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8942b0-2a81-4bb1-b569-fe814f107630 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:28.991734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8942b0-2a81-4bb1-b569-fe814f107630 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:28.991734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8942b0-2a81-4bb1-b569-fe814f107630 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:28.993047 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 672/1320] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:28.998933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceab89-39ea-4b21-9372-70a9ac596faa None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:29.000233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceab89-39ea-4b21-9372-70a9ac596faa None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:29.000423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceab89-39ea-4b21-9372-70a9ac596faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:29.049789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:29.079482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-70ceab89-39ea-4b21-9372-70a9ac596faa None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:29.081081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-70ceab89-39ea-4b21-9372-70a9ac596faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:29.191411 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 649/1321] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:29.490301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8942b0-2a81-4bb1-b569-fe814f107630 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:29.492801 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bf8942b0-2a81-4bb1-b569-fe814f107630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:29.616493 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8942b0-2a81-4bb1-b569-fe814f107630 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:29.705621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a497ac-6fdb-47b1-9da0-eaa739be3239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:29.706245 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a497ac-6fdb-47b1-9da0-eaa739be3239 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:29.706549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a497ac-6fdb-47b1-9da0-eaa739be3239 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:29.706823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a497ac-6fdb-47b1-9da0-eaa739be3239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:29.708489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-93a497ac-6fdb-47b1-9da0-eaa739be3239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:29.717052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:29.832939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a497ac-6fdb-47b1-9da0-eaa739be3239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:29.834399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a497ac-6fdb-47b1-9da0-eaa739be3239 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:29.855486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 673/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:17:33.379060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07f75f-3b00-4bdb-b254-994de7db70c1 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:33.379776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07f75f-3b00-4bdb-b254-994de7db70c1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:33.380052 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07f75f-3b00-4bdb-b254-994de7db70c1 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:33.382643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 650/1323] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:33.389089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8af3b-505e-4bab-819d-13fcf1b8934a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:33.391305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8af3b-505e-4bab-819d-13fcf1b8934a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:33.391305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8af3b-505e-4bab-819d-13fcf1b8934a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:33.479438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-06d8af3b-505e-4bab-819d-13fcf1b8934a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:33.480841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-06d8af3b-505e-4bab-819d-13fcf1b8934a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:33.592793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:33.658014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 674/1324] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:33.960452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07f75f-3b00-4bdb-b254-994de7db70c1 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:33.962353 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fb07f75f-3b00-4bdb-b254-994de7db70c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:34.142303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07f75f-3b00-4bdb-b254-994de7db70c1 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:34.159637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:34.248080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf317b-15e1-4697-892a-5f251e312286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:34.248698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf317b-15e1-4697-892a-5f251e312286 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:34.248973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf317b-15e1-4697-892a-5f251e312286 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:34.249683 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf317b-15e1-4697-892a-5f251e312286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:34.251013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-29bf317b-15e1-4697-892a-5f251e312286 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:34.410881 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf317b-15e1-4697-892a-5f251e312286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:34.412409 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf317b-15e1-4697-892a-5f251e312286 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:34.434447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 651/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:17:36.618048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c857bb-75c4-48a0-beb3-0ff9388bcbde None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:36.618743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c857bb-75c4-48a0-beb3-0ff9388bcbde None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:36.618743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c857bb-75c4-48a0-beb3-0ff9388bcbde None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:36.620320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 675/1326] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:36.627128 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df550d-0d30-4372-af85-2465dd3f8d41 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:36.627820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df550d-0d30-4372-af85-2465dd3f8d41 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:36.628110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df550d-0d30-4372-af85-2465dd3f8d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:36.686430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:36.710452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a8df550d-0d30-4372-af85-2465dd3f8d41 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:36.711836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a8df550d-0d30-4372-af85-2465dd3f8d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:36.835781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 652/1327] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:37.136612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c857bb-75c4-48a0-beb3-0ff9388bcbde None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:37.138372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-05c857bb-75c4-48a0-beb3-0ff9388bcbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:37.342939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c857bb-75c4-48a0-beb3-0ff9388bcbde None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:37.475549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cada71-272c-4389-a509-81b729363e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:37.476307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cada71-272c-4389-a509-81b729363e52 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:37.476679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cada71-272c-4389-a509-81b729363e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:37.477022 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cada71-272c-4389-a509-81b729363e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:37.478908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-18cada71-272c-4389-a509-81b729363e52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:37.503024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:37.652357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cada71-272c-4389-a509-81b729363e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:37.655331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cada71-272c-4389-a509-81b729363e52 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:37.679173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 676/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:17:39.907495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1da0b4-538f-487c-83b1-112596f0d702 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:39.908333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1da0b4-538f-487c-83b1-112596f0d702 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:39.908491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1da0b4-538f-487c-83b1-112596f0d702 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:39.910072 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 653/1329] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:39.916744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9ae02-ba74-4c2a-97bd-3524fa812e85 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:39.917335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9ae02-ba74-4c2a-97bd-3524fa812e85 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:39.917335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9ae02-ba74-4c2a-97bd-3524fa812e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:39.998278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-b0d9ae02-ba74-4c2a-97bd-3524fa812e85 None None] MFA Rules not processed for user `734f29275e5547fcb788acba34124924`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:40.000143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b0d9ae02-ba74-4c2a-97bd-3524fa812e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:40.083780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:40.123286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 677/1330] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:40.444957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1da0b4-538f-487c-83b1-112596f0d702 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:40.446783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3f1da0b4-538f-487c-83b1-112596f0d702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:40.609041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1da0b4-538f-487c-83b1-112596f0d702 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:40.648577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:40.751008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d251a-85d4-4bd5-a946-a9e9f17f988c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:40.752094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d251a-85d4-4bd5-a946-a9e9f17f988c service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:40.752582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d251a-85d4-4bd5-a946-a9e9f17f988c service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:40.753065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d251a-85d4-4bd5-a946-a9e9f17f988c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:40.756533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5a8d251a-85d4-4bd5-a946-a9e9f17f988c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:40.916740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d251a-85d4-4bd5-a946-a9e9f17f988c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:40.919718 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d251a-85d4-4bd5-a946-a9e9f17f988c service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:40.952278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 654/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:17:43.745224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf96ac-1265-4611-ae7b-0f4c7b3bc97b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:43.745854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf96ac-1265-4611-ae7b-0f4c7b3bc97b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:43.746175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf96ac-1265-4611-ae7b-0f4c7b3bc97b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:43.749389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 678/1332] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:43.757852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d965be2-bc89-4cfb-9e10-9c4d7f22814b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:43.758467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d965be2-bc89-4cfb-9e10-9c4d7f22814b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:43.758848 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d965be2-bc89-4cfb-9e10-9c4d7f22814b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:43.855430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:43.860896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-8d965be2-bc89-4cfb-9e10-9c4d7f22814b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:43.862289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8d965be2-bc89-4cfb-9e10-9c4d7f22814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:43.987341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 655/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:44.286743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf96ac-1265-4611-ae7b-0f4c7b3bc97b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:44.289465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d5bf96ac-1265-4611-ae7b-0f4c7b3bc97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:44.456877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf96ac-1265-4611-ae7b-0f4c7b3bc97b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:44.576824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbded2-4b24-49bc-8f5a-b4f05bfd800a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:44.577841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbded2-4b24-49bc-8f5a-b4f05bfd800a service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:44.578182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbded2-4b24-49bc-8f5a-b4f05bfd800a service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:44.578669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbded2-4b24-49bc-8f5a-b4f05bfd800a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:44.580493 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bdfbded2-4b24-49bc-8f5a-b4f05bfd800a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:44.616167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:44.755681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbded2-4b24-49bc-8f5a-b4f05bfd800a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:44.757552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbded2-4b24-49bc-8f5a-b4f05bfd800a service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:44.780804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 679/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:17:46.962072 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10fe4d-e2fd-4380-9c5a-8153d1362bce None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:46.962718 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10fe4d-e2fd-4380-9c5a-8153d1362bce None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:46.962718 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10fe4d-e2fd-4380-9c5a-8153d1362bce None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:46.964310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 656/1335] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:46.971178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08655-6298-47f6-8168-5de8cd49bd66 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:46.971819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08655-6298-47f6-8168-5de8cd49bd66 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:46.971819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08655-6298-47f6-8168-5de8cd49bd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:47.055922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d8c08655-6298-47f6-8168-5de8cd49bd66 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:47.057377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d8c08655-6298-47f6-8168-5de8cd49bd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:47.151091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:47.187990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 680/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:47.503615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10fe4d-e2fd-4380-9c5a-8153d1362bce None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:47.505691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5e10fe4d-e2fd-4380-9c5a-8153d1362bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:47.642521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10fe4d-e2fd-4380-9c5a-8153d1362bce None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:47.679530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:47.738922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bc34a-bd44-4151-b748-3103bf59241b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:47.739558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc34a-bd44-4151-b748-3103bf59241b service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:47.739832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc34a-bd44-4151-b748-3103bf59241b service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:47.740098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc34a-bd44-4151-b748-3103bf59241b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:47.742009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-276bc34a-bd44-4151-b748-3103bf59241b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:47.915486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bc34a-bd44-4151-b748-3103bf59241b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:47.918215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bc34a-bd44-4151-b748-3103bf59241b service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:47.957934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 657/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:17:50.895412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8c107-7603-445a-b8be-7b9eb484a28a None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:50.895955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8c107-7603-445a-b8be-7b9eb484a28a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:50.896091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8c107-7603-445a-b8be-7b9eb484a28a None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:50.898120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 681/1338] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:17:50.906503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d3be0-78b9-484c-b644-f11da637478a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:50.907371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d3be0-78b9-484c-b644-f11da637478a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:50.907371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d3be0-78b9-484c-b644-f11da637478a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:50.990287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-476d3be0-78b9-484c-b644-f11da637478a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:50.991596 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-476d3be0-78b9-484c-b644-f11da637478a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:51.001550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:51.114625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 658/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:51.408145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8c107-7603-445a-b8be-7b9eb484a28a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:51.409835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-42b8c107-7603-445a-b8be-7b9eb484a28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:51.565027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8c107-7603-445a-b8be-7b9eb484a28a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:51.698020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79ff5f-5467-4d02-97cc-4569d8087ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:51.698732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79ff5f-5467-4d02-97cc-4569d8087ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:51.699116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79ff5f-5467-4d02-97cc-4569d8087ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:51.699482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79ff5f-5467-4d02-97cc-4569d8087ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:51.701864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5a79ff5f-5467-4d02-97cc-4569d8087ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:51.748284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:51.839915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79ff5f-5467-4d02-97cc-4569d8087ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:51.841401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79ff5f-5467-4d02-97cc-4569d8087ea3 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:51.865239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 682/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:17:56.573988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241a719-cc1d-4c56-a483-05a3f95ee14b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:56.574687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241a719-cc1d-4c56-a483-05a3f95ee14b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:56.574687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241a719-cc1d-4c56-a483-05a3f95ee14b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:56.576306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 659/1341] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:17:56.582864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824992-3f32-42ec-8653-0c2342bf8451 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:56.583726 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824992-3f32-42ec-8653-0c2342bf8451 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:56.583726 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824992-3f32-42ec-8653-0c2342bf8451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:56.668796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-14824992-3f32-42ec-8653-0c2342bf8451 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:17:56.670186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-14824992-3f32-42ec-8653-0c2342bf8451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:56.801054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:56.821413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 683/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:17:57.152561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241a719-cc1d-4c56-a483-05a3f95ee14b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:17:57.155387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1241a719-cc1d-4c56-a483-05a3f95ee14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:57.326708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241a719-cc1d-4c56-a483-05a3f95ee14b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:17:57.404462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:17:57.477119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8a609-c7d4-422c-94de-2011c216c080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:17:57.477796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8a609-c7d4-422c-94de-2011c216c080 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:17:57.478176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8a609-c7d4-422c-94de-2011c216c080 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:17:57.478645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8a609-c7d4-422c-94de-2011c216c080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:17:57.481060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1de8a609-c7d4-422c-94de-2011c216c080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:17:57.649112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8a609-c7d4-422c-94de-2011c216c080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:17:57.651928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8a609-c7d4-422c-94de-2011c216c080 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:17:57.691974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 660/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:18:01.171804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f5d0-9f89-4c0b-8e42-50ba352e7d03 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:01.172400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f5d0-9f89-4c0b-8e42-50ba352e7d03 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:01.172400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f5d0-9f89-4c0b-8e42-50ba352e7d03 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:01.176830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 684/1344] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:01.187546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741240c0-40c4-4d88-a002-4518ed84a646 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:01.190066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741240c0-40c4-4d88-a002-4518ed84a646 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:01.190066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741240c0-40c4-4d88-a002-4518ed84a646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:01.279467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-741240c0-40c4-4d88-a002-4518ed84a646 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:01.280749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-741240c0-40c4-4d88-a002-4518ed84a646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:01.304711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:01.439653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 661/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:01.740413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177f5d0-9f89-4c0b-8e42-50ba352e7d03 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:01.742718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b177f5d0-9f89-4c0b-8e42-50ba352e7d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:01.881527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177f5d0-9f89-4c0b-8e42-50ba352e7d03 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:01.995333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47c635-b06f-4394-974d-e5e97f4f1f29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:01.996297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47c635-b06f-4394-974d-e5e97f4f1f29 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:01.996788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47c635-b06f-4394-974d-e5e97f4f1f29 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:01.997827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47c635-b06f-4394-974d-e5e97f4f1f29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:02.001189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3b47c635-b06f-4394-974d-e5e97f4f1f29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:02.095500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:02.199737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47c635-b06f-4394-974d-e5e97f4f1f29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:02.203575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47c635-b06f-4394-974d-e5e97f4f1f29 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:02.242295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 685/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:18:07.799016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e012829-3ad6-4506-b9a4-3b242e6d20d0 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:07.799576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e012829-3ad6-4506-b9a4-3b242e6d20d0 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:07.800025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e012829-3ad6-4506-b9a4-3b242e6d20d0 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:07.801900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 662/1347] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:07.808559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c378-a1f7-4dc3-84c0-77db0f08eccb None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:07.809175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c378-a1f7-4dc3-84c0-77db0f08eccb None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:07.809175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c378-a1f7-4dc3-84c0-77db0f08eccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:07.895565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-9af3c378-a1f7-4dc3-84c0-77db0f08eccb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:07.896830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9af3c378-a1f7-4dc3-84c0-77db0f08eccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:08.025085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:08.030682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 686/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:08.323127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e012829-3ad6-4506-b9a4-3b242e6d20d0 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:08.324722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2e012829-3ad6-4506-b9a4-3b242e6d20d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:08.479877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e012829-3ad6-4506-b9a4-3b242e6d20d0 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:08.543749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:08.599914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130e70b-cfab-480e-87d0-da67a72bcf6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:08.601198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e70b-cfab-480e-87d0-da67a72bcf6a service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:08.601652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e70b-cfab-480e-87d0-da67a72bcf6a service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:08.602054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e70b-cfab-480e-87d0-da67a72bcf6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:08.604196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5130e70b-cfab-480e-87d0-da67a72bcf6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:08.776006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130e70b-cfab-480e-87d0-da67a72bcf6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:08.777658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130e70b-cfab-480e-87d0-da67a72bcf6a service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:08.805534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 663/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:18:14.908679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85be659-cf2a-415e-b5be-1c35d47224a8 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:14.909494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85be659-cf2a-415e-b5be-1c35d47224a8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:14.909494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85be659-cf2a-415e-b5be-1c35d47224a8 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:14.911603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 687/1350] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:18:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:14.921808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4f692-8f5a-415b-9d67-626d351d7ecf None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:14.931311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4f692-8f5a-415b-9d67-626d351d7ecf None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:14.931311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4f692-8f5a-415b-9d67-626d351d7ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:15.081370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-bef4f692-8f5a-415b-9d67-626d351d7ecf None None] MFA Rules not processed for user `26b5021285c44db9b05f9d15ceaf797f`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:15.083527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bef4f692-8f5a-415b-9d67-626d351d7ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:15.118936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:15.298957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 664/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:18:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:18:21.770095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c860c-c760-45f4-a8c3-d29f0b6a4bfc None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:21.770928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c860c-c760-45f4-a8c3-d29f0b6a4bfc None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:21.772877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c860c-c760-45f4-a8c3-d29f0b6a4bfc None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:21.775590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 688/1352] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:18:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:21.786153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede1a20-eddd-4250-90ef-e8a81c36d3ff None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:21.787204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede1a20-eddd-4250-90ef-e8a81c36d3ff None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:21.787606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede1a20-eddd-4250-90ef-e8a81c36d3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:21.953103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-eede1a20-eddd-4250-90ef-e8a81c36d3ff None None] MFA Rules not processed for user `26b5021285c44db9b05f9d15ceaf797f`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:21.955349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-eede1a20-eddd-4250-90ef-e8a81c36d3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:22.122825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:22.203221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 665/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:18:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:23.992730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841027f6-8159-4c39-99c8-1474eb23ff18 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:23.993596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841027f6-8159-4c39-99c8-1474eb23ff18 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:23.993596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841027f6-8159-4c39-99c8-1474eb23ff18 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:23.994714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 689/1354] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:24.005876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b323205-9133-4383-85e4-672493f55cb9 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:24.006947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b323205-9133-4383-85e4-672493f55cb9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:24.007580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b323205-9133-4383-85e4-672493f55cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:24.125134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-5b323205-9133-4383-85e4-672493f55cb9 None None] MFA Rules not processed for user `26b5021285c44db9b05f9d15ceaf797f`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:24.126438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5b323205-9133-4383-85e4-672493f55cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:24.328980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 666/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:18:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:27.776300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96616e-3561-49a0-9b9b-5290554888e8 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:27.776912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96616e-3561-49a0-9b9b-5290554888e8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:27.776912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96616e-3561-49a0-9b9b-5290554888e8 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:27.779918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 690/1356] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:27.786191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cf0be-2c99-415d-8755-3201910016a3 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:27.786862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cf0be-2c99-415d-8755-3201910016a3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:27.786862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cf0be-2c99-415d-8755-3201910016a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:27.845051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:27.903011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-fe5cf0be-2c99-415d-8755-3201910016a3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:27.907154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fe5cf0be-2c99-415d-8755-3201910016a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:28.034954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 667/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:28.356944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a61ec8-dbe2-4e63-af75-cdc0e43658cc None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:28.356944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a61ec8-dbe2-4e63-af75-cdc0e43658cc None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:28.356944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a61ec8-dbe2-4e63-af75-cdc0e43658cc None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:28.359226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 691/1358] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 22 17:18:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:28.371545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec905333-25e0-4fc9-9817-355ad287e832 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:28.372016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec905333-25e0-4fc9-9817-355ad287e832 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:28.372398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec905333-25e0-4fc9-9817-355ad287e832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:28.478228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ec905333-25e0-4fc9-9817-355ad287e832 None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:28.478228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ec905333-25e0-4fc9-9817-355ad287e832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:28.554886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:28.679096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 668/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 22 17:18:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:28.697192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a61ec8-dbe2-4e63-af75-cdc0e43658cc None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:28.701814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d7a61ec8-dbe2-4e63-af75-cdc0e43658cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:29.012544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a61ec8-dbe2-4e63-af75-cdc0e43658cc None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:29.181794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:29.271570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038ef67-66f0-4b84-b71d-f382ab7768ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:29.272520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038ef67-66f0-4b84-b71d-f382ab7768ac service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:29.273033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038ef67-66f0-4b84-b71d-f382ab7768ac service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:29.273523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038ef67-66f0-4b84-b71d-f382ab7768ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:29.277289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7038ef67-66f0-4b84-b71d-f382ab7768ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:29.429057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038ef67-66f0-4b84-b71d-f382ab7768ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:29.432403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038ef67-66f0-4b84-b71d-f382ab7768ac service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:29.471524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 692/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 775 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:18:32.348795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aa4d7-2bde-43c6-bdd5-15f88f528e65 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:32.349681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aa4d7-2bde-43c6-bdd5-15f88f528e65 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:32.349681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aa4d7-2bde-43c6-bdd5-15f88f528e65 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:32.351200 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 669/1361] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:32.357198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278595eb-07ef-42e3-b64b-ae599f6709b2 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:32.357992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278595eb-07ef-42e3-b64b-ae599f6709b2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:32.357992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278595eb-07ef-42e3-b64b-ae599f6709b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:32.499271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-278595eb-07ef-42e3-b64b-ae599f6709b2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:32.502060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-278595eb-07ef-42e3-b64b-ae599f6709b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:32.586305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:32.725761 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 693/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:18:33.029003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3aa4d7-2bde-43c6-bdd5-15f88f528e65 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:33.031969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ea3aa4d7-2bde-43c6-bdd5-15f88f528e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:33.381127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3aa4d7-2bde-43c6-bdd5-15f88f528e65 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:33.401589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:33.639811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4e287-1e23-4a63-83a3-3db669c80f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:33.640965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4e287-1e23-4a63-83a3-3db669c80f07 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:33.642202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4e287-1e23-4a63-83a3-3db669c80f07 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:33.642735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4e287-1e23-4a63-83a3-3db669c80f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:33.646639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d3e4e287-1e23-4a63-83a3-3db669c80f07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:33.880550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4e287-1e23-4a63-83a3-3db669c80f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:33.884027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4e287-1e23-4a63-83a3-3db669c80f07 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:33.947380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 670/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 920 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:18:36.839644 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83549582-ca46-4a4c-9bca-772a36928f54 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:36.840473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83549582-ca46-4a4c-9bca-772a36928f54 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:36.840473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83549582-ca46-4a4c-9bca-772a36928f54 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:36.841862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 694/1364] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:36.847797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d8137-1491-4658-869d-427fd449b9e7 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:36.848350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d8137-1491-4658-869d-427fd449b9e7 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:36.848350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d8137-1491-4658-869d-427fd449b9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:36.867995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:36.933904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-002d8137-1491-4658-869d-427fd449b9e7 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:36.935237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-002d8137-1491-4658-869d-427fd449b9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:37.056033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 671/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:37.363636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746b1a3-3138-4b69-bdda-2f260a5d6b81 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:37.364301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746b1a3-3138-4b69-bdda-2f260a5d6b81 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:37.364873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746b1a3-3138-4b69-bdda-2f260a5d6b81 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:37.368043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 695/1366] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 22 17:18:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:37.375114 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc67c0b-1879-4e98-a1dd-a7021726a306 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:37.375378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc67c0b-1879-4e98-a1dd-a7021726a306 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:37.376943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc67c0b-1879-4e98-a1dd-a7021726a306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:37.481237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-bdc67c0b-1879-4e98-a1dd-a7021726a306 None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:37.483010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bdc67c0b-1879-4e98-a1dd-a7021726a306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:37.529692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:37.650665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 672/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 22 17:18:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:18:37.659219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746b1a3-3138-4b69-bdda-2f260a5d6b81 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:37.662208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8746b1a3-3138-4b69-bdda-2f260a5d6b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:37.904429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746b1a3-3138-4b69-bdda-2f260a5d6b81 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:37.928043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:38.044666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c268ff-f319-4edc-8139-ba36d9ef64c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:38.046214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c268ff-f319-4edc-8139-ba36d9ef64c7 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:38.046728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c268ff-f319-4edc-8139-ba36d9ef64c7 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:38.047201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c268ff-f319-4edc-8139-ba36d9ef64c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:38.051122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f5c268ff-f319-4edc-8139-ba36d9ef64c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:38.193220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c268ff-f319-4edc-8139-ba36d9ef64c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:38.195398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c268ff-f319-4edc-8139-ba36d9ef64c7 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:38.221993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 696/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:18:38.715777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87a63-b11c-4e31-bf9d-292083d93d58 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:38.716377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87a63-b11c-4e31-bf9d-292083d93d58 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:38.716377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87a63-b11c-4e31-bf9d-292083d93d58 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:38.717985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 673/1369] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:38.723649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb5b6a-e834-4c72-81e9-f916f5405da8 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:38.723950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb5b6a-e834-4c72-81e9-f916f5405da8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:38.724226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb5b6a-e834-4c72-81e9-f916f5405da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:38.775018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:38.831170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-4ccb5b6a-e834-4c72-81e9-f916f5405da8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:38.832866 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4ccb5b6a-e834-4c72-81e9-f916f5405da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:38.964961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 697/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:39.666615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5229034-484c-4865-aa86-6ee9b41c1501 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:39.667695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5229034-484c-4865-aa86-6ee9b41c1501 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:39.668116 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5229034-484c-4865-aa86-6ee9b41c1501 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:39.671142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 674/1371] 10.222.0.20 () {54 vars in 964 bytes} [Mon Aug 22 17:18:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:39.677090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d920c3-2cfb-413a-8b59-ae85c0a68847 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:39.677514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d920c3-2cfb-413a-8b59-ae85c0a68847 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:39.677701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d920c3-2cfb-413a-8b59-ae85c0a68847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:39.793747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-79d920c3-2cfb-413a-8b59-ae85c0a68847 None None] MFA Rules not processed for user `68088e59a2c64ff2948e0549f1fc7cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:39.795919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-79d920c3-2cfb-413a-8b59-ae85c0a68847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:39.928142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:39.977774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 698/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Aug 22 17:18:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:18:39.994112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5229034-484c-4865-aa86-6ee9b41c1501 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:39.997816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b5229034-484c-4865-aa86-6ee9b41c1501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:40.165065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5229034-484c-4865-aa86-6ee9b41c1501 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:40.260394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:40.324841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d836b05-da30-499c-85ad-66bea7e753aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:40.326018 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d836b05-da30-499c-85ad-66bea7e753aa service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:40.326528 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d836b05-da30-499c-85ad-66bea7e753aa service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:40.327026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d836b05-da30-499c-85ad-66bea7e753aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:40.330390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5d836b05-da30-499c-85ad-66bea7e753aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:40.586327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d836b05-da30-499c-85ad-66bea7e753aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:40.587931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d836b05-da30-499c-85ad-66bea7e753aa service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:40.615618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 675/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:18:41.247909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88576a1-a303-47fc-8c5a-a9a636f4a8ff None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:41.248804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88576a1-a303-47fc-8c5a-a9a636f4a8ff None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:41.248804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88576a1-a303-47fc-8c5a-a9a636f4a8ff None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:41.251230 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 699/1374] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:41.259523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50345-3c82-4796-b6b5-6220f6c0887f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:41.260317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50345-3c82-4796-b6b5-6220f6c0887f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:41.260317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50345-3c82-4796-b6b5-6220f6c0887f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:41.358182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-0da50345-3c82-4796-b6b5-6220f6c0887f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:41.359717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0da50345-3c82-4796-b6b5-6220f6c0887f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:41.387500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:41.492155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 676/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:41.818369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88576a1-a303-47fc-8c5a-a9a636f4a8ff None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:41.820390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c88576a1-a303-47fc-8c5a-a9a636f4a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:42.029040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88576a1-a303-47fc-8c5a-a9a636f4a8ff None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:42.098364 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:42.217289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a795fe-795c-4bb5-8348-a07f864789d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:42.218068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a795fe-795c-4bb5-8348-a07f864789d8 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:42.218447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a795fe-795c-4bb5-8348-a07f864789d8 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:42.218963 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a795fe-795c-4bb5-8348-a07f864789d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:42.221207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-83a795fe-795c-4bb5-8348-a07f864789d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:42.339423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a795fe-795c-4bb5-8348-a07f864789d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:42.341825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a795fe-795c-4bb5-8348-a07f864789d8 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:42.363229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 700/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:18:44.901781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65c240-2473-483f-b813-4a120687b823 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:44.904327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65c240-2473-483f-b813-4a120687b823 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:44.904720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65c240-2473-483f-b813-4a120687b823 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:44.906576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 677/1377] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:44.912847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e98d5b-1330-4f5b-9e95-45ea537961de None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:44.913477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e98d5b-1330-4f5b-9e95-45ea537961de None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:44.913477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e98d5b-1330-4f5b-9e95-45ea537961de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:44.996124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-58e98d5b-1330-4f5b-9e95-45ea537961de None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:44.997749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-58e98d5b-1330-4f5b-9e95-45ea537961de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:45.006190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:45.162750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 701/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:45.504615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65c240-2473-483f-b813-4a120687b823 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:45.506214 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bf65c240-2473-483f-b813-4a120687b823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:45.785434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65c240-2473-483f-b813-4a120687b823 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:45.793932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:45.945762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac2701-2a87-4c65-9c31-90a78d21e183 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:45.946598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac2701-2a87-4c65-9c31-90a78d21e183 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:45.946882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac2701-2a87-4c65-9c31-90a78d21e183 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:45.947152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac2701-2a87-4c65-9c31-90a78d21e183 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:45.948858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-39ac2701-2a87-4c65-9c31-90a78d21e183 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:46.080332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac2701-2a87-4c65-9c31-90a78d21e183 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:46.083315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac2701-2a87-4c65-9c31-90a78d21e183 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:46.116490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 678/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:18:48.618557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e03542-8be3-4d87-b656-e36b0092624f None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:48.618557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e03542-8be3-4d87-b656-e36b0092624f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:48.619447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e03542-8be3-4d87-b656-e36b0092624f None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:48.628711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 702/1380] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:48 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:48.630367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c66fb5-58f7-44df-896a-9b042f96d648 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:48.631334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c66fb5-58f7-44df-896a-9b042f96d648 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:48.631334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c66fb5-58f7-44df-896a-9b042f96d648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:48.646759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:48.757897 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-21c66fb5-58f7-44df-896a-9b042f96d648 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:48.760821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-21c66fb5-58f7-44df-896a-9b042f96d648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:48.944267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 679/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:49.271540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e03542-8be3-4d87-b656-e36b0092624f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:49.273257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-15e03542-8be3-4d87-b656-e36b0092624f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:49.458428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:49.473448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e03542-8be3-4d87-b656-e36b0092624f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:49.660029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3c82b-302a-4ac1-a071-c048c41fcacd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:49.661153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3c82b-302a-4ac1-a071-c048c41fcacd service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:49.661554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3c82b-302a-4ac1-a071-c048c41fcacd service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:49.661897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3c82b-302a-4ac1-a071-c048c41fcacd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:49.664180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-78a3c82b-302a-4ac1-a071-c048c41fcacd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:49.781169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3c82b-302a-4ac1-a071-c048c41fcacd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:49.782871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3c82b-302a-4ac1-a071-c048c41fcacd service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:49.796184 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:49.806386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 703/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:18:52.312259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548d7c-8ce3-43fa-a551-dad5f7ea8e49 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:52.313073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548d7c-8ce3-43fa-a551-dad5f7ea8e49 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:52.313268 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548d7c-8ce3-43fa-a551-dad5f7ea8e49 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:52.315202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 680/1383] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:52.323024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46de9a-4ce3-4b05-99a7-311d1ef2915b None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:52.323592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46de9a-4ce3-4b05-99a7-311d1ef2915b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:52.323592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46de9a-4ce3-4b05-99a7-311d1ef2915b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:52.412531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-4c46de9a-4ce3-4b05-99a7-311d1ef2915b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:52.414100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4c46de9a-4ce3-4b05-99a7-311d1ef2915b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:52.611911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 704/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:52.918278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d548d7c-8ce3-43fa-a551-dad5f7ea8e49 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:52.920888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8d548d7c-8ce3-43fa-a551-dad5f7ea8e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:53.068175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:53.368922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d548d7c-8ce3-43fa-a551-dad5f7ea8e49 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:53.585547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bfd63-7999-4bae-af6f-00bf03ea211f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:53.586521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bfd63-7999-4bae-af6f-00bf03ea211f service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:53.586859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bfd63-7999-4bae-af6f-00bf03ea211f service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:53.587223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bfd63-7999-4bae-af6f-00bf03ea211f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:53.590324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4f1bfd63-7999-4bae-af6f-00bf03ea211f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:53.672681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:53.752865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bfd63-7999-4bae-af6f-00bf03ea211f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:53.754907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bfd63-7999-4bae-af6f-00bf03ea211f service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:53.778665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 681/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 861 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:18:56.372850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c258017-1c40-4c62-a8e8-cdbbded15b7d None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:56.373499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c258017-1c40-4c62-a8e8-cdbbded15b7d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:56.374158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c258017-1c40-4c62-a8e8-cdbbded15b7d None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:56.378016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 705/1386] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:56.387523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b7385-85e6-4106-9faa-feb4cc7e6972 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:56.388098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b7385-85e6-4106-9faa-feb4cc7e6972 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:56.388098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b7385-85e6-4106-9faa-feb4cc7e6972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:56.469373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c78b7385-85e6-4106-9faa-feb4cc7e6972 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:56.470676 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c78b7385-85e6-4106-9faa-feb4cc7e6972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:56.597075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 682/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:18:56.938047 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c258017-1c40-4c62-a8e8-cdbbded15b7d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:56.940028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0c258017-1c40-4c62-a8e8-cdbbded15b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:57.057818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:57.161119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c258017-1c40-4c62-a8e8-cdbbded15b7d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:57.336340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb7763-ac6b-4ccc-b09d-a6226994a3b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:57.336879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb7763-ac6b-4ccc-b09d-a6226994a3b7 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:57.337158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb7763-ac6b-4ccc-b09d-a6226994a3b7 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:57.337429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb7763-ac6b-4ccc-b09d-a6226994a3b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:57.339301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-80cb7763-ac6b-4ccc-b09d-a6226994a3b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:57.390042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:57.458897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb7763-ac6b-4ccc-b09d-a6226994a3b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:57.465294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb7763-ac6b-4ccc-b09d-a6226994a3b7 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:57.488678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 706/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:18:58.685220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce467f42-ad0c-42ba-977c-1b0695c72d01 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:58.686125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce467f42-ad0c-42ba-977c-1b0695c72d01 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:58.686125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce467f42-ad0c-42ba-977c-1b0695c72d01 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:58.687537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 683/1389] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:18:58.693660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b33e5-a513-42ad-9193-a54712a1ce6a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:58.694671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b33e5-a513-42ad-9193-a54712a1ce6a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:58.694671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b33e5-a513-42ad-9193-a54712a1ce6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:58.782847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-3e7b33e5-a513-42ad-9193-a54712a1ce6a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:58.784935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3e7b33e5-a513-42ad-9193-a54712a1ce6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:58.962169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 707/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:18:59.277437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7c92d-f4cf-46c4-aa72-647094a2607b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:59.277929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7c92d-f4cf-46c4-aa72-647094a2607b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:59.278347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7c92d-f4cf-46c4-aa72-647094a2607b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:59.280158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 684/1391] 10.222.0.20 () {54 vars in 967 bytes} [Mon Aug 22 17:18:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:18:59.288866 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f89645-26f0-42b2-a36f-6ce021232d50 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:59.289257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f89645-26f0-42b2-a36f-6ce021232d50 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:59.289684 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f89645-26f0-42b2-a36f-6ce021232d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:59.307783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:59.413586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-a4f89645-26f0-42b2-a36f-6ce021232d50 None None] MFA Rules not processed for user `ea4c2870e2754f3188228fc441950df2`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:18:59.415027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a4f89645-26f0-42b2-a36f-6ce021232d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:59.524753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 708/1392] 10.222.0.20 () {58 vars in 1064 bytes} [Mon Aug 22 17:18:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:18:59.534748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7c92d-f4cf-46c4-aa72-647094a2607b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:18:59.536560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7bd7c92d-f4cf-46c4-aa72-647094a2607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:59.574735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:59.671267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7c92d-f4cf-46c4-aa72-647094a2607b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:18:59.765877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4bc67-0af2-4610-b70a-556aa8648f0e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea4c2870e2754f3188228fc441950df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:18:59.766969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4bc67-0af2-4610-b70a-556aa8648f0e service glance] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:18:59.766969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4bc67-0af2-4610-b70a-556aa8648f0e service glance] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:18:59.766969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4bc67-0af2-4610-b70a-556aa8648f0e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:18:59.768693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cda4bc67-0af2-4610-b70a-556aa8648f0e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:18:59.888594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:18:59.948367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4bc67-0af2-4610-b70a-556aa8648f0e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:18:59.950023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4bc67-0af2-4610-b70a-556aa8648f0e service glance] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:18:59.975489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 685/1393] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:00.042245 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2f729-295e-4a9c-9336-9daa8ea165ee None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:00.042906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2f729-295e-4a9c-9336-9daa8ea165ee None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:00.042906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2f729-295e-4a9c-9336-9daa8ea165ee None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:00.044466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 709/1394] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:00.050608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805d49-a2c4-440f-b90c-396f267c4464 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:00.052403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805d49-a2c4-440f-b90c-396f267c4464 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:00.052403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805d49-a2c4-440f-b90c-396f267c4464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:00.152684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-98805d49-a2c4-440f-b90c-396f267c4464 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:00.154739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-98805d49-a2c4-440f-b90c-396f267c4464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:00.318593 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 686/1395] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:19:00.689899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2f729-295e-4a9c-9336-9daa8ea165ee None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:00.692666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f5b2f729-295e-4a9c-9336-9daa8ea165ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:00.857394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:00.938280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2f729-295e-4a9c-9336-9daa8ea165ee None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:01.098445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb8526-2ddd-45b8-976b-cb24f239746a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:01.099332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb8526-2ddd-45b8-976b-cb24f239746a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:01.099550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb8526-2ddd-45b8-976b-cb24f239746a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:01.127406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95e33c-8454-472c-acf6-673cd682c68c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:01.128303 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95e33c-8454-472c-acf6-673cd682c68c service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:01.128691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95e33c-8454-472c-acf6-673cd682c68c service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:01.129065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95e33c-8454-472c-acf6-673cd682c68c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:01.131283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4a95e33c-8454-472c-acf6-673cd682c68c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:01.162079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:01.199652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-e4eb8526-2ddd-45b8-976b-cb24f239746a None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:01.201847 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e4eb8526-2ddd-45b8-976b-cb24f239746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:01.248836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:01.282605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95e33c-8454-472c-acf6-673cd682c68c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:01.286218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95e33c-8454-472c-acf6-673cd682c68c service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:01.317873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 710/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:01.334161 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 687/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:19:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:01.397689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd0518-831b-4daa-8be6-f69e7de7fb3a None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:01.401675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd0518-831b-4daa-8be6-f69e7de7fb3a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:01.401675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd0518-831b-4daa-8be6-f69e7de7fb3a None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:01.401675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 711/1398] 10.222.0.20 () {54 vars in 943 bytes} [Mon Aug 22 17:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:01.415337 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7816226-01c9-4c40-b26c-3ff8a65d8d67 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:01.415833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7816226-01c9-4c40-b26c-3ff8a65d8d67 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:01.416281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7816226-01c9-4c40-b26c-3ff8a65d8d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:01.550349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a7816226-01c9-4c40-b26c-3ff8a65d8d67 None None] MFA Rules not processed for user `38462b1165514ed7b948cba710782d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:01.551692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a7816226-01c9-4c40-b26c-3ff8a65d8d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:01.662035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:01.731319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 688/1399] 10.222.0.20 () {58 vars in 1040 bytes} [Mon Aug 22 17:19:01 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:01.749219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd0518-831b-4daa-8be6-f69e7de7fb3a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:01.753446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-83bd0518-831b-4daa-8be6-f69e7de7fb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:01.997831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd0518-831b-4daa-8be6-f69e7de7fb3a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:02.083537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:02.172537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d30b9-c6cb-4c69-92a4-c528badfeec4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:02.173584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d30b9-c6cb-4c69-92a4-c528badfeec4 service swift] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:02.173584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d30b9-c6cb-4c69-92a4-c528badfeec4 service swift] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:02.173911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d30b9-c6cb-4c69-92a4-c528badfeec4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:02.177656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3c6d30b9-c6cb-4c69-92a4-c528badfeec4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:02.341240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d30b9-c6cb-4c69-92a4-c528badfeec4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:02.343940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d30b9-c6cb-4c69-92a4-c528badfeec4 service swift] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:02.358276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 712/1400] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:19:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 17:19:03.739402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7be29-5b6d-46b9-9f27-764803ec3fe9 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:03.740364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7be29-5b6d-46b9-9f27-764803ec3fe9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:03.740546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7be29-5b6d-46b9-9f27-764803ec3fe9 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:03.750394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 689/1401] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:19:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:03.751622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc0ddb-a55e-4c5a-b8a1-edddcc1934b8 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:03.752188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc0ddb-a55e-4c5a-b8a1-edddcc1934b8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:03.752578 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc0ddb-a55e-4c5a-b8a1-edddcc1934b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:03.893748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-14bc0ddb-a55e-4c5a-b8a1-edddcc1934b8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:03.895818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-14bc0ddb-a55e-4c5a-b8a1-edddcc1934b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:03.992296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:04.094132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 713/1402] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:04.450041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7be29-5b6d-46b9-9f27-764803ec3fe9 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:04.451771 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9ad7be29-5b6d-46b9-9f27-764803ec3fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:04.655134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7be29-5b6d-46b9-9f27-764803ec3fe9 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:04.733147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:04.887895 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917def6-b150-4f05-9f63-8423053d7760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:04.888855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917def6-b150-4f05-9f63-8423053d7760 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:04.889439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917def6-b150-4f05-9f63-8423053d7760 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:04.889903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917def6-b150-4f05-9f63-8423053d7760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:04.893215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f917def6-b150-4f05-9f63-8423053d7760 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:05.050624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917def6-b150-4f05-9f63-8423053d7760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:05.053672 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917def6-b150-4f05-9f63-8423053d7760 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:05.081206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 690/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:07.478972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada1db5-f69a-4d9c-922f-12cd5f195f3c None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:07.479560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada1db5-f69a-4d9c-922f-12cd5f195f3c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:07.479898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada1db5-f69a-4d9c-922f-12cd5f195f3c None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:07.482453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 714/1404] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:19:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:19:07.489532 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5f833-d9c9-4c07-af47-ecf59cc1b2c6 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:07.490537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5f833-d9c9-4c07-af47-ecf59cc1b2c6 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:07.491459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5f833-d9c9-4c07-af47-ecf59cc1b2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:07.589316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:07.601447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-bbd5f833-d9c9-4c07-af47-ecf59cc1b2c6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:07.603699 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bbd5f833-d9c9-4c07-af47-ecf59cc1b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:07.829940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 691/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:19:08.393710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada1db5-f69a-4d9c-922f-12cd5f195f3c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:08.396514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9ada1db5-f69a-4d9c-922f-12cd5f195f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:08.722022 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada1db5-f69a-4d9c-922f-12cd5f195f3c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:08.762327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:08.957062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32efba-bf57-4797-98f3-a4a8875c016d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:08.957934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32efba-bf57-4797-98f3-a4a8875c016d service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:08.958322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32efba-bf57-4797-98f3-a4a8875c016d service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:08.958922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32efba-bf57-4797-98f3-a4a8875c016d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:08.966698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0a32efba-bf57-4797-98f3-a4a8875c016d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:09.144442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32efba-bf57-4797-98f3-a4a8875c016d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:09.147514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32efba-bf57-4797-98f3-a4a8875c016d service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:09.188054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 715/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 795 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:19:12.529304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fe65d-a147-4308-904d-584302ac6ed5 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:12.529896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fe65d-a147-4308-904d-584302ac6ed5 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:12.529896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fe65d-a147-4308-904d-584302ac6ed5 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:12.531386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 692/1407] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:12.538390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de3cad-360e-4bc9-b39b-fe6b8723f0eb None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:12.538988 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de3cad-360e-4bc9-b39b-fe6b8723f0eb None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:12.539560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de3cad-360e-4bc9-b39b-fe6b8723f0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:12.593468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:12.637694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c7de3cad-360e-4bc9-b39b-fe6b8723f0eb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:12.639088 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c7de3cad-360e-4bc9-b39b-fe6b8723f0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:12.773645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 716/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:13.109145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fe65d-a147-4308-904d-584302ac6ed5 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:13.111598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-319fe65d-a147-4308-904d-584302ac6ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:13.303156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:13.342089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fe65d-a147-4308-904d-584302ac6ed5 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:13.528502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd1cd9-2472-46ef-8702-f40b483383ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:13.529677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd1cd9-2472-46ef-8702-f40b483383ef service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:13.530358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd1cd9-2472-46ef-8702-f40b483383ef service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:13.530358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd1cd9-2472-46ef-8702-f40b483383ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:13.532242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-80dd1cd9-2472-46ef-8702-f40b483383ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:13.647816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:13.653377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd1cd9-2472-46ef-8702-f40b483383ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:13.656405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd1cd9-2472-46ef-8702-f40b483383ef service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:13.679817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 693/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:22.771488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=52 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:23.684849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=7 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:30.292972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e1fdf-24c7-4e9b-94f7-a364b9d5f258 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:30.293592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e1fdf-24c7-4e9b-94f7-a364b9d5f258 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:30.293876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e1fdf-24c7-4e9b-94f7-a364b9d5f258 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:30.296394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 717/1410] 10.222.0.20 () {54 vars in 918 bytes} [Mon Aug 22 17:19:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:30.304394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3160d-a9a4-4d08-badc-68edc6ed524a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:30.305589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3160d-a9a4-4d08-badc-68edc6ed524a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:30.305589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3160d-a9a4-4d08-badc-68edc6ed524a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:30.417909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-61f3160d-a9a4-4d08-badc-68edc6ed524a None None] MFA Rules not processed for user `26b5021285c44db9b05f9d15ceaf797f`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:30.419183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-61f3160d-a9a4-4d08-badc-68edc6ed524a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:30.548451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 694/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Aug 22 17:19:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:30.564127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d9038-5581-44ea-91d9-bb353d73219a None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:30.564575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d9038-5581-44ea-91d9-bb353d73219a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:30.564958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d9038-5581-44ea-91d9-bb353d73219a None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:30.566705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 718/1412] 10.222.0.20 () {54 vars in 918 bytes} [Mon Aug 22 17:19:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:19:30.573252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e5cc-b0c3-43f8-a8ae-783bf12983fd None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:30.573598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e5cc-b0c3-43f8-a8ae-783bf12983fd None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:30.574046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e5cc-b0c3-43f8-a8ae-783bf12983fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:30.665584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:30.669101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-5aa3e5cc-b0c3-43f8-a8ae-783bf12983fd None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:30.670394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5aa3e5cc-b0c3-43f8-a8ae-783bf12983fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:30.811854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 695/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Aug 22 17:19:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:33.362617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701124-1d82-4989-94a4-0405b222156d None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:33.363471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701124-1d82-4989-94a4-0405b222156d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:33.363471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701124-1d82-4989-94a4-0405b222156d None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:33.364853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:19:33.372044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbac0e-2979-4faf-a058-6ea6da7b3c3d None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:33.372629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbac0e-2979-4faf-a058-6ea6da7b3c3d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:33.372738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbac0e-2979-4faf-a058-6ea6da7b3c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:33.495900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-9bbbac0e-2979-4faf-a058-6ea6da7b3c3d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:33.498071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9bbbac0e-2979-4faf-a058-6ea6da7b3c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:33.564083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:33.614977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:33.912101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1701124-1d82-4989-94a4-0405b222156d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:33.913649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e1701124-1d82-4989-94a4-0405b222156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:34.096533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1701124-1d82-4989-94a4-0405b222156d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:34.184097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:34.231342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5cebc-bffa-4ca3-be33-b8ea9aaeeadf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:34.231966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5cebc-bffa-4ca3-be33-b8ea9aaeeadf service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:34.232243 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5cebc-bffa-4ca3-be33-b8ea9aaeeadf service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:34.232510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5cebc-bffa-4ca3-be33-b8ea9aaeeadf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:34.234216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f0b5cebc-bffa-4ca3-be33-b8ea9aaeeadf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:34.357549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5cebc-bffa-4ca3-be33-b8ea9aaeeadf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:34.360524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5cebc-bffa-4ca3-be33-b8ea9aaeeadf service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:34.385910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 720/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:35.350335 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4af67-6d87-412e-be89-f652bcd4cd00 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:35.350977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4af67-6d87-412e-be89-f652bcd4cd00 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:35.351076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4af67-6d87-412e-be89-f652bcd4cd00 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:35.352810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:35.358847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269c812-2230-4761-bf5f-3915f01474f8 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:35.359183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269c812-2230-4761-bf5f-3915f01474f8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:35.359497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269c812-2230-4761-bf5f-3915f01474f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:35.454082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-a269c812-2230-4761-bf5f-3915f01474f8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:35.455475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a269c812-2230-4761-bf5f-3915f01474f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:35.488916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:35.573478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:35.731550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4af67-6d87-412e-be89-f652bcd4cd00 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:35.733155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1ce4af67-6d87-412e-be89-f652bcd4cd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:35.987009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4af67-6d87-412e-be89-f652bcd4cd00 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:36.038564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:36.153097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f700b6-3fb8-481d-ba11-bf20513355ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:36.153865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f700b6-3fb8-481d-ba11-bf20513355ac service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:36.154248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f700b6-3fb8-481d-ba11-bf20513355ac service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:36.154776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f700b6-3fb8-481d-ba11-bf20513355ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:36.156893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-78f700b6-3fb8-481d-ba11-bf20513355ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:36.275066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f700b6-3fb8-481d-ba11-bf20513355ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:36.277096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f700b6-3fb8-481d-ba11-bf20513355ac service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:36.300559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 698/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:19:37.257085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f7407-3db1-48f9-8cdb-52f2a30f24fe None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:37.258000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f7407-3db1-48f9-8cdb-52f2a30f24fe None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:37.258000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f7407-3db1-48f9-8cdb-52f2a30f24fe None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:37.260319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:19:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:19:37.266246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6586cd-3be3-4c97-bca3-69cca9a6e54d None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:37.266862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6586cd-3be3-4c97-bca3-69cca9a6e54d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:37.266862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6586cd-3be3-4c97-bca3-69cca9a6e54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:37.332899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:37.360017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2b6586cd-3be3-4c97-bca3-69cca9a6e54d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:37.361254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2b6586cd-3be3-4c97-bca3-69cca9a6e54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:37.481226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:37.645378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f7407-3db1-48f9-8cdb-52f2a30f24fe None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:37.650284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-256f7407-3db1-48f9-8cdb-52f2a30f24fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:37.842256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f7407-3db1-48f9-8cdb-52f2a30f24fe None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:37.857533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:37.984693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0854f-fadb-448f-9726-3c49a70cde92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:37.985308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0854f-fadb-448f-9726-3c49a70cde92 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:37.985580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0854f-fadb-448f-9726-3c49a70cde92 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:37.985848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0854f-fadb-448f-9726-3c49a70cde92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:37.987589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dbd0854f-fadb-448f-9726-3c49a70cde92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:38.102351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0854f-fadb-448f-9726-3c49a70cde92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:38.104051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0854f-fadb-448f-9726-3c49a70cde92 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:38.126954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 723/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:39.094664 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b70785-d479-44d7-bb05-b4951f038c5e None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:39.095630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b70785-d479-44d7-bb05-b4951f038c5e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:39.095630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b70785-d479-44d7-bb05-b4951f038c5e None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:39.096560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:39.102938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750d0ec-9874-4800-93d9-66f583c216a6 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:39.103803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750d0ec-9874-4800-93d9-66f583c216a6 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:39.103803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750d0ec-9874-4800-93d9-66f583c216a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:39.127429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:39.199099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2750d0ec-9874-4800-93d9-66f583c216a6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:39.200410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2750d0ec-9874-4800-93d9-66f583c216a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:39.319334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:19:39.476289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b70785-d479-44d7-bb05-b4951f038c5e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:39.481161 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-80b70785-d479-44d7-bb05-b4951f038c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:39.623561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:39.668085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b70785-d479-44d7-bb05-b4951f038c5e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:39.806465 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9e802-8292-4ce0-be25-e7384c52589f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:39.807054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9e802-8292-4ce0-be25-e7384c52589f service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:39.807326 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9e802-8292-4ce0-be25-e7384c52589f service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:39.807640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9e802-8292-4ce0-be25-e7384c52589f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:39.809468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c2f9e802-8292-4ce0-be25-e7384c52589f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:39.914379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:39.931775 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9e802-8292-4ce0-be25-e7384c52589f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:39.933483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9e802-8292-4ce0-be25-e7384c52589f service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:39.956155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 701/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:40.888440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba46d4-49f8-454a-9048-d513a682552b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:40.889278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba46d4-49f8-454a-9048-d513a682552b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:40.889278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba46d4-49f8-454a-9048-d513a682552b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:40.890781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:19:40.896654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a40b0b-a11a-4c2d-88ce-302e6b626cc8 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:40.897388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a40b0b-a11a-4c2d-88ce-302e6b626cc8 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:40.897388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a40b0b-a11a-4c2d-88ce-302e6b626cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:40.993380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d2a40b0b-a11a-4c2d-88ce-302e6b626cc8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:40.994877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d2a40b0b-a11a-4c2d-88ce-302e6b626cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:41.120037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:19:41.277878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba46d4-49f8-454a-9048-d513a682552b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:41.279537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a2ba46d4-49f8-454a-9048-d513a682552b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:41.379593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:41.490551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba46d4-49f8-454a-9048-d513a682552b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:41.651182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36336afb-2a11-416b-9a33-7d10362cd569 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:41.651804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36336afb-2a11-416b-9a33-7d10362cd569 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:41.652094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36336afb-2a11-416b-9a33-7d10362cd569 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:41.652360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36336afb-2a11-416b-9a33-7d10362cd569 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:41.654082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-36336afb-2a11-416b-9a33-7d10362cd569 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:41.716190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:41.765323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36336afb-2a11-416b-9a33-7d10362cd569 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:41.767273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36336afb-2a11-416b-9a33-7d10362cd569 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:41.790556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 726/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:19:42.751879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6e80e-6551-41e9-936c-c9001d4addba None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:42.752537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6e80e-6551-41e9-936c-c9001d4addba None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:42.752537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6e80e-6551-41e9-936c-c9001d4addba None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:42.753771 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:19:42.760326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93bcac-3947-407f-b449-5b9afeb20f24 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:42.760896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93bcac-3947-407f-b449-5b9afeb20f24 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:42.761134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93bcac-3947-407f-b449-5b9afeb20f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:42.856964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-fd93bcac-3947-407f-b449-5b9afeb20f24 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:42.858300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fd93bcac-3947-407f-b449-5b9afeb20f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:42.984310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:19:43.141377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6e80e-6551-41e9-936c-c9001d4addba None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:43.142972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cfe6e80e-6551-41e9-936c-c9001d4addba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:43.201565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:43.360201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6e80e-6551-41e9-936c-c9001d4addba None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:43.547290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955a8d6-b024-48e2-805f-8ebd6e9e1384 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:43.548265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955a8d6-b024-48e2-805f-8ebd6e9e1384 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:43.548720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955a8d6-b024-48e2-805f-8ebd6e9e1384 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:43.549142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955a8d6-b024-48e2-805f-8ebd6e9e1384 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:43.552345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f955a8d6-b024-48e2-805f-8ebd6e9e1384 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:43.566184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:43.720747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955a8d6-b024-48e2-805f-8ebd6e9e1384 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:43.723978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955a8d6-b024-48e2-805f-8ebd6e9e1384 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:43.761766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 704/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:44.688465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23d6c-90ab-40a5-9b3b-952dbca865bd None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:44.689140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23d6c-90ab-40a5-9b3b-952dbca865bd None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:44.689140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23d6c-90ab-40a5-9b3b-952dbca865bd None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:44.690389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:44.697032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52501a0-3302-411e-b296-b128912b280b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:44.697339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52501a0-3302-411e-b296-b128912b280b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:44.697575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52501a0-3302-411e-b296-b128912b280b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:44.793426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b52501a0-3302-411e-b296-b128912b280b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:44.794773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b52501a0-3302-411e-b296-b128912b280b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:44.879735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:44.915990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:19:45.073043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f23d6c-90ab-40a5-9b3b-952dbca865bd None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:45.074653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e9f23d6c-90ab-40a5-9b3b-952dbca865bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:45.095600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:45.313206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f23d6c-90ab-40a5-9b3b-952dbca865bd None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:45.423476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:45.455699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83329b-f9fc-45cb-9bb4-bf306eb31674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:45.456575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83329b-f9fc-45cb-9bb4-bf306eb31674 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:45.456911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83329b-f9fc-45cb-9bb4-bf306eb31674 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:45.457320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83329b-f9fc-45cb-9bb4-bf306eb31674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:45.459520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fa83329b-f9fc-45cb-9bb4-bf306eb31674 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:45.597785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83329b-f9fc-45cb-9bb4-bf306eb31674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:45.599384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83329b-f9fc-45cb-9bb4-bf306eb31674 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:45.623287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 729/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:19:46.547374 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466263-7146-4e46-8431-c5fb13d39add None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:46.548292 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466263-7146-4e46-8431-c5fb13d39add None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:46.548292 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466263-7146-4e46-8431-c5fb13d39add None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:46.549248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:46.555733 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8341325-f692-42fc-b580-24fe8cc21d2a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:46.556722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8341325-f692-42fc-b580-24fe8cc21d2a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:46.556722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8341325-f692-42fc-b580-24fe8cc21d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:46.654280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f8341325-f692-42fc-b580-24fe8cc21d2a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:46.655548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f8341325-f692-42fc-b580-24fe8cc21d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:46.706000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:46.778751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:46.940040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466263-7146-4e46-8431-c5fb13d39add None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:46.943015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-52466263-7146-4e46-8431-c5fb13d39add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:47.141903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466263-7146-4e46-8431-c5fb13d39add None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:47.215249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:47.331927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d6313-76de-4aec-a92e-4663e9c99f23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:47.332973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d6313-76de-4aec-a92e-4663e9c99f23 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:47.333446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d6313-76de-4aec-a92e-4663e9c99f23 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:47.333922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d6313-76de-4aec-a92e-4663e9c99f23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:47.338672 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-084d6313-76de-4aec-a92e-4663e9c99f23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:47.457252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d6313-76de-4aec-a92e-4663e9c99f23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:47.460329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d6313-76de-4aec-a92e-4663e9c99f23 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:47.496509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 707/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:48.435391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6797be-729c-424e-9dc4-60815cee356a None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:48.436036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6797be-729c-424e-9dc4-60815cee356a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:48.436363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6797be-729c-424e-9dc4-60815cee356a None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:48.439084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:19:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:48.445098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd90960-01ce-41af-9af3-436a5d991f32 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:48.445676 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd90960-01ce-41af-9af3-436a5d991f32 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:48.445676 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd90960-01ce-41af-9af3-436a5d991f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:48.547309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2bd90960-01ce-41af-9af3-436a5d991f32 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:48.548584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2bd90960-01ce-41af-9af3-436a5d991f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:48.551833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:48.676439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:48.842639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6797be-729c-424e-9dc4-60815cee356a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:48.844413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7f6797be-729c-424e-9dc4-60815cee356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:49.048392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6797be-729c-424e-9dc4-60815cee356a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:49.105465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:49.236206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874d055-9430-41a7-b03a-1e0fd4411d05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:49.236836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874d055-9430-41a7-b03a-1e0fd4411d05 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:49.237115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874d055-9430-41a7-b03a-1e0fd4411d05 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:49.237388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874d055-9430-41a7-b03a-1e0fd4411d05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:49.239436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a874d055-9430-41a7-b03a-1e0fd4411d05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:49.356457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874d055-9430-41a7-b03a-1e0fd4411d05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:49.358079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874d055-9430-41a7-b03a-1e0fd4411d05 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:49.380925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 732/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:19:51.798867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0165ee-9a88-43c3-b1d9-45bd008b2e60 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:51.799455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0165ee-9a88-43c3-b1d9-45bd008b2e60 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:51.799455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0165ee-9a88-43c3-b1d9-45bd008b2e60 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:51.801217 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:51.809011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f62d-0f3d-4809-a990-59e7cb5c6d2f None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:51.809669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f62d-0f3d-4809-a990-59e7cb5c6d2f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:51.809669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f62d-0f3d-4809-a990-59e7cb5c6d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:51.860950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:51.892764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-20f2f62d-0f3d-4809-a990-59e7cb5c6d2f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:51.893937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-20f2f62d-0f3d-4809-a990-59e7cb5c6d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:52.046393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:52.344622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0165ee-9a88-43c3-b1d9-45bd008b2e60 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:52.346456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2f0165ee-9a88-43c3-b1d9-45bd008b2e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:52.472923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0165ee-9a88-43c3-b1d9-45bd008b2e60 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:52.519725 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:52.573856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d700d54-fe03-445e-b588-07b3436740fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:52.574424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d700d54-fe03-445e-b588-07b3436740fb service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:52.574855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d700d54-fe03-445e-b588-07b3436740fb service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:52.575184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d700d54-fe03-445e-b588-07b3436740fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:52.578270 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2d700d54-fe03-445e-b588-07b3436740fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:52.716986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d700d54-fe03-445e-b588-07b3436740fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:52.718757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d700d54-fe03-445e-b588-07b3436740fb service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:52.741657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 710/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:55.021245 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1300d2-6b38-40d5-980e-e751800e0ed4 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:55.021911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1300d2-6b38-40d5-980e-e751800e0ed4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:55.021911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1300d2-6b38-40d5-980e-e751800e0ed4 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:55.024133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:55.030773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf928f34-b763-4973-a2df-c4d479c19491 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:55.031522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf928f34-b763-4973-a2df-c4d479c19491 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:55.031522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf928f34-b763-4973-a2df-c4d479c19491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:55.114086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-cf928f34-b763-4973-a2df-c4d479c19491 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:55.115464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cf928f34-b763-4973-a2df-c4d479c19491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:55.132866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:55.239471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:55.542698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1300d2-6b38-40d5-980e-e751800e0ed4 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:55.544448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ad1300d2-6b38-40d5-980e-e751800e0ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:55.680622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1300d2-6b38-40d5-980e-e751800e0ed4 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:55.696363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:55.780143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84599e62-3edb-4093-8bb4-b375559cdf8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:55.780776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84599e62-3edb-4093-8bb4-b375559cdf8a service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:55.781056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84599e62-3edb-4093-8bb4-b375559cdf8a service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:55.781328 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84599e62-3edb-4093-8bb4-b375559cdf8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:55.783366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-84599e62-3edb-4093-8bb4-b375559cdf8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:55.941346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84599e62-3edb-4093-8bb4-b375559cdf8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:55.945897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84599e62-3edb-4093-8bb4-b375559cdf8a service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:55.974629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 735/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:19:58.661973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44fb1e-ab0c-46a5-ba33-f4aa35813a14 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:58.662941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44fb1e-ab0c-46a5-ba33-f4aa35813a14 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:58.662941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44fb1e-ab0c-46a5-ba33-f4aa35813a14 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:58.665211 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:19:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:19:58.671698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad18084-6b73-4220-85b5-594b7dd177bd None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:58.672507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad18084-6b73-4220-85b5-594b7dd177bd None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:58.672667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad18084-6b73-4220-85b5-594b7dd177bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:58.735559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:58.759810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-9ad18084-6b73-4220-85b5-594b7dd177bd None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:19:58.761121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9ad18084-6b73-4220-85b5-594b7dd177bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:58.885121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:19:59.181321 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44fb1e-ab0c-46a5-ba33-f4aa35813a14 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:19:59.183066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4d44fb1e-ab0c-46a5-ba33-f4aa35813a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:59.310825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44fb1e-ab0c-46a5-ba33-f4aa35813a14 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:19:59.363101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:19:59.439499 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cfc7e-ddbc-41c0-9607-9c0a34df1d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:19:59.440455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cfc7e-ddbc-41c0-9607-9c0a34df1d68 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:19:59.440948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cfc7e-ddbc-41c0-9607-9c0a34df1d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:19:59.441430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cfc7e-ddbc-41c0-9607-9c0a34df1d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:19:59.446309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-388cfc7e-ddbc-41c0-9607-9c0a34df1d68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:19:59.595208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388cfc7e-ddbc-41c0-9607-9c0a34df1d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:19:59.596705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388cfc7e-ddbc-41c0-9607-9c0a34df1d68 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:19:59.620905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 713/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:20:01.919428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1322e05-e28f-46bd-8d21-19a7316ad9eb None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:01.920000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1322e05-e28f-46bd-8d21-19a7316ad9eb None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:01.920387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1322e05-e28f-46bd-8d21-19a7316ad9eb None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:01.922977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:20:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:20:01.930225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16accafe-c84a-4a85-a0c1-973318109e08 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:01.930823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16accafe-c84a-4a85-a0c1-973318109e08 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:01.930823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16accafe-c84a-4a85-a0c1-973318109e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:02.010596 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-16accafe-c84a-4a85-a0c1-973318109e08 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:02.011870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-16accafe-c84a-4a85-a0c1-973318109e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:02.039321 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:02.156138 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:02.450432 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1322e05-e28f-46bd-8d21-19a7316ad9eb None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:02.452219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a1322e05-e28f-46bd-8d21-19a7316ad9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:02.586005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1322e05-e28f-46bd-8d21-19a7316ad9eb None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:02.611074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:02.683738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed16fc6-9bb4-4a0f-bc2f-ca38ba5145e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:02.684378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed16fc6-9bb4-4a0f-bc2f-ca38ba5145e2 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:02.684660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed16fc6-9bb4-4a0f-bc2f-ca38ba5145e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:02.684931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed16fc6-9bb4-4a0f-bc2f-ca38ba5145e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:02.686708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-eed16fc6-9bb4-4a0f-bc2f-ca38ba5145e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:02.822107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed16fc6-9bb4-4a0f-bc2f-ca38ba5145e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:02.823641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed16fc6-9bb4-4a0f-bc2f-ca38ba5145e2 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:02.846641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 738/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:20:05.138225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ae40d-ac0f-4394-bdd5-f830b49f8baa None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:05.138932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ae40d-ac0f-4394-bdd5-f830b49f8baa None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:05.138932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ae40d-ac0f-4394-bdd5-f830b49f8baa None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:05.140247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:20:05.146736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dff12-085e-4214-a628-d5f5f68bfbb7 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:05.147976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dff12-085e-4214-a628-d5f5f68bfbb7 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:05.148086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dff12-085e-4214-a628-d5f5f68bfbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:05.224133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:05.227975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-bc5dff12-085e-4214-a628-d5f5f68bfbb7 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:05.229231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bc5dff12-085e-4214-a628-d5f5f68bfbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:05.379354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:20:05.679061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ae40d-ac0f-4394-bdd5-f830b49f8baa None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:05.680742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-459ae40d-ac0f-4394-bdd5-f830b49f8baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:05.806957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ae40d-ac0f-4394-bdd5-f830b49f8baa None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:05.901564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:05.958119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d7a42-3d9b-4a23-92ac-d057e57db1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:05.959234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d7a42-3d9b-4a23-92ac-d057e57db1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:05.961151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d7a42-3d9b-4a23-92ac-d057e57db1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:05.961655 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d7a42-3d9b-4a23-92ac-d057e57db1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:05.964975 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-022d7a42-3d9b-4a23-92ac-d057e57db1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:06.105155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d7a42-3d9b-4a23-92ac-d057e57db1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:06.106674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d7a42-3d9b-4a23-92ac-d057e57db1b0 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:06.129946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 716/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:20:08.339026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf28a91-7f26-4e5f-904a-06b3d17a0305 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:08.339829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf28a91-7f26-4e5f-904a-06b3d17a0305 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:08.339829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf28a91-7f26-4e5f-904a-06b3d17a0305 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:08.341207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:20:08.347669 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eca07b-d800-46e0-bbb7-19202fd9e09f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:08.348255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eca07b-d800-46e0-bbb7-19202fd9e09f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:08.348255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eca07b-d800-46e0-bbb7-19202fd9e09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:08.428273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-85eca07b-d800-46e0-bbb7-19202fd9e09f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:08.429553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-85eca07b-d800-46e0-bbb7-19202fd9e09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:08.464015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:08.549028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:08.845291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf28a91-7f26-4e5f-904a-06b3d17a0305 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:08.847145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ddf28a91-7f26-4e5f-904a-06b3d17a0305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:08.976903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf28a91-7f26-4e5f-904a-06b3d17a0305 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:09.014872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:09.074221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a511af-3e24-4b64-a220-842fc779d3c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:09.074956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a511af-3e24-4b64-a220-842fc779d3c7 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:09.075277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a511af-3e24-4b64-a220-842fc779d3c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:09.075594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a511af-3e24-4b64-a220-842fc779d3c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:09.077381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f4a511af-3e24-4b64-a220-842fc779d3c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:09.214603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a511af-3e24-4b64-a220-842fc779d3c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:09.216168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a511af-3e24-4b64-a220-842fc779d3c7 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:09.246908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 741/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:20:11.360093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a5975-9e71-428c-b12b-31d01d7e661c None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:11.360799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a5975-9e71-428c-b12b-31d01d7e661c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:11.360799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a5975-9e71-428c-b12b-31d01d7e661c None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:11.362032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:20:11.370649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875774e5-610d-41f4-a797-53b2cfb1631d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:11.371399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875774e5-610d-41f4-a797-53b2cfb1631d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:11.371799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875774e5-610d-41f4-a797-53b2cfb1631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:11.480178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-875774e5-610d-41f4-a797-53b2cfb1631d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:11.481445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-875774e5-610d-41f4-a797-53b2cfb1631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:11.492717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:11.603125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:11.892422 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a5975-9e71-428c-b12b-31d01d7e661c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:11.894242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2d0a5975-9e71-428c-b12b-31d01d7e661c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:12.021355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a5975-9e71-428c-b12b-31d01d7e661c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:12.094819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:12.117165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c925b6b-eaa0-4cde-bc06-cd188d7f3a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:12.117801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c925b6b-eaa0-4cde-bc06-cd188d7f3a70 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:12.118091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c925b6b-eaa0-4cde-bc06-cd188d7f3a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:12.118357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c925b6b-eaa0-4cde-bc06-cd188d7f3a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:12.120151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4c925b6b-eaa0-4cde-bc06-cd188d7f3a70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:12.250529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c925b6b-eaa0-4cde-bc06-cd188d7f3a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:12.252027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c925b6b-eaa0-4cde-bc06-cd188d7f3a70 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:12.275462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 719/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:20:14.532336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577420-a144-478f-a3ee-dc6655632e5e None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:14.533057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577420-a144-478f-a3ee-dc6655632e5e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:14.533057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577420-a144-478f-a3ee-dc6655632e5e None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:14.534594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:20:14.542347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb58df8-7500-4836-a64a-dd33a55f8691 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:14.543064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb58df8-7500-4836-a64a-dd33a55f8691 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:14.543064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb58df8-7500-4836-a64a-dd33a55f8691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:14.629208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-0cb58df8-7500-4836-a64a-dd33a55f8691 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:14.630653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0cb58df8-7500-4836-a64a-dd33a55f8691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:14.679580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:14.751929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:15.045142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90577420-a144-478f-a3ee-dc6655632e5e None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:15.046997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-90577420-a144-478f-a3ee-dc6655632e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:15.220990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90577420-a144-478f-a3ee-dc6655632e5e None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:15.279954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:15.328239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362e002-6c32-4a24-b6ec-bf1ae638cdad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:15.328777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362e002-6c32-4a24-b6ec-bf1ae638cdad service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:15.329063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362e002-6c32-4a24-b6ec-bf1ae638cdad service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:15.329328 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362e002-6c32-4a24-b6ec-bf1ae638cdad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:15.331087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5362e002-6c32-4a24-b6ec-bf1ae638cdad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:15.598267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362e002-6c32-4a24-b6ec-bf1ae638cdad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:15.600600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362e002-6c32-4a24-b6ec-bf1ae638cdad service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:15.625260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 744/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:20:17.812712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e56133-b102-4006-93b1-b8c94aab6348 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:17.813554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e56133-b102-4006-93b1-b8c94aab6348 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:17.813554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e56133-b102-4006-93b1-b8c94aab6348 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:17.814698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:20:17.820981 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638634e-cf7f-4980-b32f-04759e7bf3e2 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:17.821534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638634e-cf7f-4980-b32f-04759e7bf3e2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:17.821534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638634e-cf7f-4980-b32f-04759e7bf3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:17.918788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-5638634e-cf7f-4980-b32f-04759e7bf3e2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:17.920191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5638634e-cf7f-4980-b32f-04759e7bf3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:17.943537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:18.072397 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:18.383046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e56133-b102-4006-93b1-b8c94aab6348 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:18.384720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-37e56133-b102-4006-93b1-b8c94aab6348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:18.515581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e56133-b102-4006-93b1-b8c94aab6348 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:18.607372 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:18.618003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdf81e-4f89-4200-99e1-80dd936ab2f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:18.619976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdf81e-4f89-4200-99e1-80dd936ab2f4 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:18.619976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdf81e-4f89-4200-99e1-80dd936ab2f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:18.619976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdf81e-4f89-4200-99e1-80dd936ab2f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:18.622092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e4bdf81e-4f89-4200-99e1-80dd936ab2f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:18.792292 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdf81e-4f89-4200-99e1-80dd936ab2f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:18.794534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdf81e-4f89-4200-99e1-80dd936ab2f4 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:18.830315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 722/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:20:20.883632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a881dd-1874-4070-b987-bd8d10bf22d7 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:20.884237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a881dd-1874-4070-b987-bd8d10bf22d7 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:20.884237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a881dd-1874-4070-b987-bd8d10bf22d7 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:20.885471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:20:20.892182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71328c5-0bae-4795-a07e-a7da40e8bf27 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:20.892974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71328c5-0bae-4795-a07e-a7da40e8bf27 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:20.892974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71328c5-0bae-4795-a07e-a7da40e8bf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:20.987889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a71328c5-0bae-4795-a07e-a7da40e8bf27 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:20.989134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a71328c5-0bae-4795-a07e-a7da40e8bf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:21.051619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:21.119971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:21.432280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a881dd-1874-4070-b987-bd8d10bf22d7 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:21.433832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f7a881dd-1874-4070-b987-bd8d10bf22d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:21.566424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a881dd-1874-4070-b987-bd8d10bf22d7 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:21.640862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:21.678997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3215e6b-fd3e-4cf8-9ff6-f345aa1ea26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:21.679719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3215e6b-fd3e-4cf8-9ff6-f345aa1ea26d service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:21.680140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3215e6b-fd3e-4cf8-9ff6-f345aa1ea26d service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:21.680673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3215e6b-fd3e-4cf8-9ff6-f345aa1ea26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:21.682855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b3215e6b-fd3e-4cf8-9ff6-f345aa1ea26d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:21.825807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3215e6b-fd3e-4cf8-9ff6-f345aa1ea26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:21.827616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3215e6b-fd3e-4cf8-9ff6-f345aa1ea26d service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:21.852171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 747/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:20:31.160302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=22 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:31.849102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=66 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:43.545004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c760ac-a080-412e-a647-6cb4ddaace17 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:43.545623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c760ac-a080-412e-a647-6cb4ddaace17 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:43.545623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c760ac-a080-412e-a647-6cb4ddaace17 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:43.547234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 724/1471] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:20:43.553860 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae3edd-0f8a-4547-8217-0ea9224de9e3 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:43.554533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae3edd-0f8a-4547-8217-0ea9224de9e3 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:43.554533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae3edd-0f8a-4547-8217-0ea9224de9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:43.643848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-99ae3edd-0f8a-4547-8217-0ea9224de9e3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:43.645666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-99ae3edd-0f8a-4547-8217-0ea9224de9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:43.790086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 748/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:44.104762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c92f4-0dcc-4ae8-8926-ab952ded76e3 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:44.105161 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c92f4-0dcc-4ae8-8926-ab952ded76e3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:44.105482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c92f4-0dcc-4ae8-8926-ab952ded76e3 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:44.107029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 725/1473] 10.222.0.20 () {54 vars in 967 bytes} [Mon Aug 22 17:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:20:44.112681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32581b7e-cfd0-474d-a37b-f4a8375400dc None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:44.113012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32581b7e-cfd0-474d-a37b-f4a8375400dc None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:44.115122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32581b7e-cfd0-474d-a37b-f4a8375400dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:44.206210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-32581b7e-cfd0-474d-a37b-f4a8375400dc None None] MFA Rules not processed for user `ea4c2870e2754f3188228fc441950df2`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:44.208165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-32581b7e-cfd0-474d-a37b-f4a8375400dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:44.233238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:44.304593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 749/1474] 10.222.0.20 () {58 vars in 1064 bytes} [Mon Aug 22 17:20:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:44.313392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c92f4-0dcc-4ae8-8926-ab952ded76e3 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:44.315300 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-635c92f4-0dcc-4ae8-8926-ab952ded76e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:44.450001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c92f4-0dcc-4ae8-8926-ab952ded76e3 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:44.570939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c0d82-a6f4-4c71-a595-dcb11375a20d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea4c2870e2754f3188228fc441950df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:44.571952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c0d82-a6f4-4c71-a595-dcb11375a20d service glance] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:44.572400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c0d82-a6f4-4c71-a595-dcb11375a20d service glance] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:44.572837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c0d82-a6f4-4c71-a595-dcb11375a20d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:44.575879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-500c0d82-a6f4-4c71-a595-dcb11375a20d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:44.691891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:44.768536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c0d82-a6f4-4c71-a595-dcb11375a20d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:44.770613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c0d82-a6f4-4c71-a595-dcb11375a20d service glance] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:44.794231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 726/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:20:46.965733 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0add2-f036-4126-beb9-356ca5c19546 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:46.966371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0add2-f036-4126-beb9-356ca5c19546 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:46.966371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0add2-f036-4126-beb9-356ca5c19546 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:46.967743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 750/1476] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:20:46.974716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86b2a-f355-4a84-9dbc-ea469d56f41c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:46.975477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86b2a-f355-4a84-9dbc-ea469d56f41c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:46.975477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86b2a-f355-4a84-9dbc-ea469d56f41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:47.067276 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ade86b2a-f355-4a84-9dbc-ea469d56f41c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:47.068886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ade86b2a-f355-4a84-9dbc-ea469d56f41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:47.198797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 727/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:47.482324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0add2-f036-4126-beb9-356ca5c19546 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:47.485150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8bd0add2-f036-4126-beb9-356ca5c19546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:47.730299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0add2-f036-4126-beb9-356ca5c19546 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:47.821450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:47.907672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e8844-492d-48e3-9d19-a811734406bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:47.908827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e8844-492d-48e3-9d19-a811734406bd admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:47.909284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e8844-492d-48e3-9d19-a811734406bd admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:47.909746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e8844-492d-48e3-9d19-a811734406bd admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:47.913813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e8844-492d-48e3-9d19-a811734406bd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:47.917890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WARNING py.warnings [None req-606e8844-492d-48e3-9d19-a811734406bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:20:47.917890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: warnings.warn(msg) Aug 22 17:20:47.917890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: Aug 22 17:20:47.920502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e8844-492d-48e3-9d19-a811734406bd admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:47.962533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 751/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 22 17:20:47 2022] GET /identity/v3/projects => generated 2553 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:20:50.037502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812a5f3-0e01-49a7-b39a-9d46cb2c6b0d None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:50.038150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812a5f3-0e01-49a7-b39a-9d46cb2c6b0d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:50.038150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812a5f3-0e01-49a7-b39a-9d46cb2c6b0d None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:50.039750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 728/1479] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:20:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:20:50.046219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eb57b-71d0-472b-a7ef-f9fb9042bbf0 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:50.047021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eb57b-71d0-472b-a7ef-f9fb9042bbf0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:50.047021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eb57b-71d0-472b-a7ef-f9fb9042bbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:50.129858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6f9eb57b-71d0-472b-a7ef-f9fb9042bbf0 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:50.131156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6f9eb57b-71d0-472b-a7ef-f9fb9042bbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:50.257403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 752/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:50.577356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812a5f3-0e01-49a7-b39a-9d46cb2c6b0d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:50.579016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e812a5f3-0e01-49a7-b39a-9d46cb2c6b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:50.590050 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:50.757162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812a5f3-0e01-49a7-b39a-9d46cb2c6b0d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:50.860963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:50.903732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023c101-2582-4c07-8527-612b24fbb398 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:50.904698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023c101-2582-4c07-8527-612b24fbb398 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:50.905403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023c101-2582-4c07-8527-612b24fbb398 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:50.906070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023c101-2582-4c07-8527-612b24fbb398 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:50.908118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0023c101-2582-4c07-8527-612b24fbb398 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:51.036207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023c101-2582-4c07-8527-612b24fbb398 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:51.037939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023c101-2582-4c07-8527-612b24fbb398 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:51.067941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 729/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:20:53.336852 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21cd7a-341c-4db6-bc70-fcb6ec4a705b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:53.337779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21cd7a-341c-4db6-bc70-fcb6ec4a705b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:53.337779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21cd7a-341c-4db6-bc70-fcb6ec4a705b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:53.338923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 753/1482] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:20:53.345916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62408013-ff7f-4adb-b070-80a2762f39d5 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:53.346535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62408013-ff7f-4adb-b070-80a2762f39d5 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:53.346947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62408013-ff7f-4adb-b070-80a2762f39d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:53.449394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-62408013-ff7f-4adb-b070-80a2762f39d5 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:53.450707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-62408013-ff7f-4adb-b070-80a2762f39d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:53.508530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:53.585419 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 730/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:20:54.049439 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21cd7a-341c-4db6-bc70-fcb6ec4a705b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:54.051421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9a21cd7a-341c-4db6-bc70-fcb6ec4a705b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:54.080646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:54.182888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21cd7a-341c-4db6-bc70-fcb6ec4a705b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:54.275275 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3fe4e-88d8-408d-b19b-081dc8b6950e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:54.275901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3fe4e-88d8-408d-b19b-081dc8b6950e service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:54.276184 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3fe4e-88d8-408d-b19b-081dc8b6950e service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:54.276455 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3fe4e-88d8-408d-b19b-081dc8b6950e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:54.278143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ffd3fe4e-88d8-408d-b19b-081dc8b6950e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:54.376532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:54.449984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3fe4e-88d8-408d-b19b-081dc8b6950e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:54.452423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3fe4e-88d8-408d-b19b-081dc8b6950e service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:54.483012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 754/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:20:56.577870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1295f-fae0-4923-95a2-258d55b17d86 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:56.578799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1295f-fae0-4923-95a2-258d55b17d86 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:56.578967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1295f-fae0-4923-95a2-258d55b17d86 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:56.580569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 731/1485] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:20:56.587507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7f046-413a-44e1-b582-d62540ed30c4 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:56.588198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7f046-413a-44e1-b582-d62540ed30c4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:56.588198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7f046-413a-44e1-b582-d62540ed30c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:56.672534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c3f7f046-413a-44e1-b582-d62540ed30c4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:20:56.673767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c3f7f046-413a-44e1-b582-d62540ed30c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:56.793053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 755/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:20:57.299455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1295f-fae0-4923-95a2-258d55b17d86 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:20:57.302381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c0b1295f-fae0-4923-95a2-258d55b17d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:57.434343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1295f-fae0-4923-95a2-258d55b17d86 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:20:57.532308 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:20:57.535219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4b686-1ba8-46de-ba7d-75e98ba6fb82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:20:57.536183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b686-1ba8-46de-ba7d-75e98ba6fb82 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:20:57.536584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b686-1ba8-46de-ba7d-75e98ba6fb82 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:20:57.536952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b686-1ba8-46de-ba7d-75e98ba6fb82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:20:57.539004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-49d4b686-1ba8-46de-ba7d-75e98ba6fb82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:20:57.681737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4b686-1ba8-46de-ba7d-75e98ba6fb82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:20:57.683860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4b686-1ba8-46de-ba7d-75e98ba6fb82 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:20:57.709651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 732/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:21:00.026124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5a1ab-d29f-4781-8ee4-87ec13583938 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:21:00.026766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5a1ab-d29f-4781-8ee4-87ec13583938 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:21:00.027121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5a1ab-d29f-4781-8ee4-87ec13583938 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:21:00.029837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 756/1488] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:21:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:21:00.037235 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9e425-2a01-4b7b-bb4c-960d4bfffa23 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:21:00.037825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9e425-2a01-4b7b-bb4c-960d4bfffa23 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:21:00.038187 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9e425-2a01-4b7b-bb4c-960d4bfffa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:21:00.129141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-23f9e425-2a01-4b7b-bb4c-960d4bfffa23 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:21:00.130453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-23f9e425-2a01-4b7b-bb4c-960d4bfffa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:21:00.196992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:21:00.249586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 733/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:21:00.553723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5a1ab-d29f-4781-8ee4-87ec13583938 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:21:00.556665 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-aad5a1ab-d29f-4781-8ee4-87ec13583938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:21:00.630031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:21:00.742677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5a1ab-d29f-4781-8ee4-87ec13583938 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:21:00.833426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406d766-01d4-4a88-abe4-23e8a7d8185e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:21:00.834039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406d766-01d4-4a88-abe4-23e8a7d8185e service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:21:00.834314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406d766-01d4-4a88-abe4-23e8a7d8185e service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:21:00.834615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406d766-01d4-4a88-abe4-23e8a7d8185e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:21:00.836285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1406d766-01d4-4a88-abe4-23e8a7d8185e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:21:00.934699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:21:00.973045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406d766-01d4-4a88-abe4-23e8a7d8185e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:21:00.974759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406d766-01d4-4a88-abe4-23e8a7d8185e service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:21:00.997672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 757/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:21:03.650416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b18b1-ccbc-488a-a8d4-89c6c57b1d9b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:21:03.651288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b18b1-ccbc-488a-a8d4-89c6c57b1d9b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:21:03.651288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b18b1-ccbc-488a-a8d4-89c6c57b1d9b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:21:03.652439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 734/1491] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 22 17:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:21:03.658771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae00195-0d6b-430d-a6f1-9e2fc4030615 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:21:03.658771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae00195-0d6b-430d-a6f1-9e2fc4030615 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:21:03.658771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae00195-0d6b-430d-a6f1-9e2fc4030615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:21:03.742946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-bae00195-0d6b-430d-a6f1-9e2fc4030615 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:21:03.744263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bae00195-0d6b-430d-a6f1-9e2fc4030615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:21:03.903327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 758/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 22 17:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:21:04.199121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b18b1-ccbc-488a-a8d4-89c6c57b1d9b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:21:04.200930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-591b18b1-ccbc-488a-a8d4-89c6c57b1d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:21:04.334564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b18b1-ccbc-488a-a8d4-89c6c57b1d9b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:21:04.426827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1339e1-3c86-4bae-a3f6-a4a8bcbc7bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:21:04.427433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1339e1-3c86-4bae-a3f6-a4a8bcbc7bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:21:04.427743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1339e1-3c86-4bae-a3f6-a4a8bcbc7bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:21:04.428023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1339e1-3c86-4bae-a3f6-a4a8bcbc7bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:21:04.429792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0b1339e1-3c86-4bae-a3f6-a4a8bcbc7bf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:21:04.441850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:21:04.585669 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1339e1-3c86-4bae-a3f6-a4a8bcbc7bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:21:04.587120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1339e1-3c86-4bae-a3f6-a4a8bcbc7bf5 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:21:04.609070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 735/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:21:13.899511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=82 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:21:14.611342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=50 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:22:35.413557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756411e-578d-4b30-87dc-fcaeb608ca7d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:35.414412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756411e-578d-4b30-87dc-fcaeb608ca7d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:35.414412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756411e-578d-4b30-87dc-fcaeb608ca7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:35.517220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-4756411e-578d-4b30-87dc-fcaeb608ca7d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:22:35.518770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4756411e-578d-4b30-87dc-fcaeb608ca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:22:35.651616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 759/1494] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:22:35.662121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ec3f6-fc1c-4a5b-a5d8-7c41ff7a0d6a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:35.662761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ec3f6-fc1c-4a5b-a5d8-7c41ff7a0d6a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:35.663152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ec3f6-fc1c-4a5b-a5d8-7c41ff7a0d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:35.700440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-553ec3f6-fc1c-4a5b-a5d8-7c41ff7a0d6a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:22:35.704017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-553ec3f6-fc1c-4a5b-a5d8-7c41ff7a0d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:22:35.720798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 736/1495] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:22:35.725879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756411e-578d-4b30-87dc-fcaeb608ca7d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:22:35.736680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756411e-578d-4b30-87dc-fcaeb608ca7d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:22:35.742255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bc8c3-34e7-4ae3-a792-71bd262f249c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:22:35.742931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc8c3-34e7-4ae3-a792-71bd262f249c admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:35.743209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc8c3-34e7-4ae3-a792-71bd262f249c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:35.743479 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc8c3-34e7-4ae3-a792-71bd262f249c admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:35.744145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bc8c3-34e7-4ae3-a792-71bd262f249c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:22:35.744447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bc8c3-34e7-4ae3-a792-71bd262f249c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:22:35.745582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WARNING py.warnings [None req-4a2bc8c3-34e7-4ae3-a792-71bd262f249c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:22:35.745582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: warnings.warn(msg) Aug 22 17:22:35.745582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: Aug 22 17:22:35.746609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bc8c3-34e7-4ae3-a792-71bd262f249c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:22:35.760875 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 760/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:22:35.771912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ec3f6-fc1c-4a5b-a5d8-7c41ff7a0d6a None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:22:35.782320 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ec3f6-fc1c-4a5b-a5d8-7c41ff7a0d6a None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:22:35.787574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec55e6a-bb9b-4649-82ba-0862ca3fc71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:22:35.788211 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55e6a-bb9b-4649-82ba-0862ca3fc71d admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:35.788489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55e6a-bb9b-4649-82ba-0862ca3fc71d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:35.788758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55e6a-bb9b-4649-82ba-0862ca3fc71d admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:35.789414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec55e6a-bb9b-4649-82ba-0862ca3fc71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:22:35.791420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: WARNING py.warnings [None req-dec55e6a-bb9b-4649-82ba-0862ca3fc71d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:22:35.791420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: warnings.warn(msg) Aug 22 17:22:35.791420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: Aug 22 17:22:35.792686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec55e6a-bb9b-4649-82ba-0862ca3fc71d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:22:35.950181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 737/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:22:35 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:22:35.956128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bc8c3-34e7-4ae3-a792-71bd262f249c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:22:35.967756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bc8c3-34e7-4ae3-a792-71bd262f249c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:22:36.027518 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:22:36.036372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f8c92-5ee6-40f2-8c5d-9744c32e7360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:22:36.037037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f8c92-5ee6-40f2-8c5d-9744c32e7360 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:36.037340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f8c92-5ee6-40f2-8c5d-9744c32e7360 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:36.037710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f8c92-5ee6-40f2-8c5d-9744c32e7360 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:36.038375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f8c92-5ee6-40f2-8c5d-9744c32e7360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:22:36.039500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WARNING py.warnings [None req-e91f8c92-5ee6-40f2-8c5d-9744c32e7360 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:22:36.039500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: warnings.warn(msg) Aug 22 17:22:36.039500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: Aug 22 17:22:36.040748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f8c92-5ee6-40f2-8c5d-9744c32e7360 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:22:36.183452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 761/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:22:35 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:22:36.190688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec55e6a-bb9b-4649-82ba-0862ca3fc71d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:22:36.207654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec55e6a-bb9b-4649-82ba-0862ca3fc71d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:22:36.213177 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd234a-bb64-456e-b9ec-b8f829cfd669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:22:36.219761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd234a-bb64-456e-b9ec-b8f829cfd669 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:36.219761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd234a-bb64-456e-b9ec-b8f829cfd669 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:36.219761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd234a-bb64-456e-b9ec-b8f829cfd669 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:36.219761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd234a-bb64-456e-b9ec-b8f829cfd669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:22:36.219761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: WARNING py.warnings [None req-a0cd234a-bb64-456e-b9ec-b8f829cfd669 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:22:36.219761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: warnings.warn(msg) Aug 22 17:22:36.219761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: Aug 22 17:22:36.220695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd234a-bb64-456e-b9ec-b8f829cfd669 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:22:36.233015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 738/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:22:36.237717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f8c92-5ee6-40f2-8c5d-9744c32e7360 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:22:36.249919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f8c92-5ee6-40f2-8c5d-9744c32e7360 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:22:36.255018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a148e6c-c9bf-47da-adf8-90fd7fdc1398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:22:36.255737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a148e6c-c9bf-47da-adf8-90fd7fdc1398 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:36.256018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a148e6c-c9bf-47da-adf8-90fd7fdc1398 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:36.256293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a148e6c-c9bf-47da-adf8-90fd7fdc1398 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:36.256955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a148e6c-c9bf-47da-adf8-90fd7fdc1398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:22:36.258003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WARNING py.warnings [None req-1a148e6c-c9bf-47da-adf8-90fd7fdc1398 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:22:36.258003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: warnings.warn(msg) Aug 22 17:22:36.258003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: Aug 22 17:22:36.258978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a148e6c-c9bf-47da-adf8-90fd7fdc1398 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:22:36.275503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 762/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:22:36.280648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd234a-bb64-456e-b9ec-b8f829cfd669 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:22:36.292130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd234a-bb64-456e-b9ec-b8f829cfd669 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:22:36.299565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519186b-2224-49e5-a470-9622141c8ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:22:36.300242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519186b-2224-49e5-a470-9622141c8ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:36.300518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519186b-2224-49e5-a470-9622141c8ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:36.300793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519186b-2224-49e5-a470-9622141c8ef8 admin admin] PATH_INFO: `/v3/projects/649ba423feec4c04a05b29381f831cb0/users/bee803e6b73a4378abd8dfe9a1ab8390/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:36.320670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519186b-2224-49e5-a470-9622141c8ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=649ba423feec4c04a05b29381f831cb0, user_id=bee803e6b73a4378abd8dfe9a1ab8390, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:22:36.321797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: WARNING py.warnings [None req-7519186b-2224-49e5-a470-9622141c8ef8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:22:36.321797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: warnings.warn(msg) Aug 22 17:22:36.321797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: Aug 22 17:22:36.324428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519186b-2224-49e5-a470-9622141c8ef8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:22:36.462087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 739/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:22:36 2022] PUT /identity/v3/projects/649ba423feec4c04a05b29381f831cb0/users/bee803e6b73a4378abd8dfe9a1ab8390/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:22:36.475302 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ac20b-0e85-49c8-b007-a976bf43811d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:36.482306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ac20b-0e85-49c8-b007-a976bf43811d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:36.482306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ac20b-0e85-49c8-b007-a976bf43811d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:36.513235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-4d1ac20b-0e85-49c8-b007-a976bf43811d None None] MFA Rules not processed for user `bee803e6b73a4378abd8dfe9a1ab8390`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:22:36.514688 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4d1ac20b-0e85-49c8-b007-a976bf43811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:22:36.619076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 763/1502] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:22:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:22:36.640761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f966a-80a9-4b59-a9a3-f99ad909515a None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:36.641210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f966a-80a9-4b59-a9a3-f99ad909515a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:36.641646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f966a-80a9-4b59-a9a3-f99ad909515a None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:36.644549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 740/1503] 10.222.0.20 () {54 vars in 964 bytes} [Mon Aug 22 17:22:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:22:36.650211 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c885a-3399-440b-904d-723c8dfec57d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:36.650552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c885a-3399-440b-904d-723c8dfec57d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:36.650856 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c885a-3399-440b-904d-723c8dfec57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:36.724327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-895c885a-3399-440b-904d-723c8dfec57d None None] MFA Rules not processed for user `68088e59a2c64ff2948e0549f1fc7cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:22:36.725484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-895c885a-3399-440b-904d-723c8dfec57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:22:36.830450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 764/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Aug 22 17:22:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:22:36.839291 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f966a-80a9-4b59-a9a3-f99ad909515a None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:22:36.858297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f966a-80a9-4b59-a9a3-f99ad909515a None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:22:36.864767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbd68e-c3ca-44b3-91bc-2bc8fa254864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:22:36.865858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbd68e-c3ca-44b3-91bc-2bc8fa254864 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:36.866319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbd68e-c3ca-44b3-91bc-2bc8fa254864 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:36.866851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbd68e-c3ca-44b3-91bc-2bc8fa254864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:36.878001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbd68e-c3ca-44b3-91bc-2bc8fa254864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:22:36.879515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbd68e-c3ca-44b3-91bc-2bc8fa254864 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:22:36.890727 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 741/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:22:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:22:36.948340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7991f-62aa-4bd5-976f-4f4c53f6a77b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:36.948698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7991f-62aa-4bd5-976f-4f4c53f6a77b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:36.949086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7991f-62aa-4bd5-976f-4f4c53f6a77b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:36.951038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 765/1506] 10.222.0.20 () {52 vars in 830 bytes} [Mon Aug 22 17:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:22:36.956051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbd68e-c3ca-44b3-91bc-2bc8fa254864 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:22:37.033387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbd68e-c3ca-44b3-91bc-2bc8fa254864 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:22:37.038523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:22:37.039193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:37.039470 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:37.039741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] PATH_INFO: `/v3/users/bee803e6b73a4378abd8dfe9a1ab8390` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:37.041025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bee803e6b73a4378abd8dfe9a1ab8390)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:22:37.042132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: WARNING py.warnings [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:22:37.042132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: warnings.warn(msg) Aug 22 17:22:37.042132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: Aug 22 17:22:37.043423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:22:37.281978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bee803e6b73a4378abd8dfe9a1ab8390'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:22:37.282840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bee803e6b73a4378abd8dfe9a1ab8390'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:22:37.283879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bee803e6b73a4378abd8dfe9a1ab8390'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:22:37.301898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bee803e6b73a4378abd8dfe9a1ab8390'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:22:37.334139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bee803e6b73a4378abd8dfe9a1ab8390'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:22:37.339426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:22:37.428842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 742/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:22:36 2022] DELETE /identity/v3/users/bee803e6b73a4378abd8dfe9a1ab8390 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:22:37.433773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aabf41-44c2-4850-9fff-a2867917c565 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:37.434197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aabf41-44c2-4850-9fff-a2867917c565 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:37.434679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aabf41-44c2-4850-9fff-a2867917c565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:37.456490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d2aabf41-44c2-4850-9fff-a2867917c565 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:22:37.457922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d2aabf41-44c2-4850-9fff-a2867917c565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:22:37.470073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:22:37.532913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 766/1508] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:22:37.544016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:22:37.545900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:22:37.662649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647489c7-e57a-49d5-a11d-23f24798bba4 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:22:37.668417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e1b42-3f7b-4346-8331-9eeb30dfb069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:22:37.670701 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e1b42-3f7b-4346-8331-9eeb30dfb069 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:37.671233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e1b42-3f7b-4346-8331-9eeb30dfb069 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:37.671722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e1b42-3f7b-4346-8331-9eeb30dfb069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:37.689878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e1b42-3f7b-4346-8331-9eeb30dfb069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:22:37.692707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e1b42-3f7b-4346-8331-9eeb30dfb069 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:22:37.712725 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 743/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:22:38.295128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aabf41-44c2-4850-9fff-a2867917c565 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:22:38.295128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d2aabf41-44c2-4850-9fff-a2867917c565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:22:38.316096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aabf41-44c2-4850-9fff-a2867917c565 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:22:38.324210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:22:38.324982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:22:38.325566 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:22:38.326570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] PATH_INFO: `/v3/projects/649ba423feec4c04a05b29381f831cb0` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:22:38.328370 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=649ba423feec4c04a05b29381f831cb0)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:22:38.330183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WARNING py.warnings [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:22:38.330183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: warnings.warn(msg) Aug 22 17:22:38.330183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: Aug 22 17:22:38.331785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:22:38.503520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '649ba423feec4c04a05b29381f831cb0', 'request_id': 'req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:22:38.518706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '649ba423feec4c04a05b29381f831cb0', 'request_id': 'req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:22:38.525986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '649ba423feec4c04a05b29381f831cb0', 'request_id': 'req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:22:38.628403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] The token cache is being invalidate because project 649ba423feec4c04a05b29381f831cb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:22:38.638752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:22:38.638752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9517ffcd-1f84-4e85-96fe-b3d0c78fa67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:22:38.638752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 767/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:22:38 2022] DELETE /identity/v3/projects/649ba423feec4c04a05b29381f831cb0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:22:47.703901 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:22:48.624507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:24:16.689154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba076e2-98f6-4e08-983a-4cff4dd57cbe None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:16.689819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba076e2-98f6-4e08-983a-4cff4dd57cbe None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:16.689819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba076e2-98f6-4e08-983a-4cff4dd57cbe None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:16.691239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:24:16.697491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702dafe-9598-42c6-895b-78fd56a8c997 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:16.698089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702dafe-9598-42c6-895b-78fd56a8c997 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:16.698089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702dafe-9598-42c6-895b-78fd56a8c997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:16.719785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-9702dafe-9598-42c6-895b-78fd56a8c997 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:24:16.721042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9702dafe-9598-42c6-895b-78fd56a8c997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:16.803274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:24:16.973843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba076e2-98f6-4e08-983a-4cff4dd57cbe None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:24:16.975702 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0ba076e2-98f6-4e08-983a-4cff4dd57cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:17.104695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba076e2-98f6-4e08-983a-4cff4dd57cbe None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:24:17.111248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740de946-e5f7-43a3-8b71-0772638fef7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:24:17.111880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740de946-e5f7-43a3-8b71-0772638fef7c service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:17.112158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740de946-e5f7-43a3-8b71-0772638fef7c service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:17.112428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740de946-e5f7-43a3-8b71-0772638fef7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:17.122750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740de946-e5f7-43a3-8b71-0772638fef7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:24:17.124400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740de946-e5f7-43a3-8b71-0772638fef7c service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:24:17.135595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 745/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:24:18.019602 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453c6c0-9ecf-47f5-a177-62080011e058 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:18.020479 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453c6c0-9ecf-47f5-a177-62080011e058 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:18.020660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453c6c0-9ecf-47f5-a177-62080011e058 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:18.022250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:24:18.029170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcce4ae-11d3-4aca-b6bb-b310b90f87b4 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:18.030132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcce4ae-11d3-4aca-b6bb-b310b90f87b4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:18.030132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcce4ae-11d3-4aca-b6bb-b310b90f87b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:18.051246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-7dcce4ae-11d3-4aca-b6bb-b310b90f87b4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:24:18.052487 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7dcce4ae-11d3-4aca-b6bb-b310b90f87b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:18.063127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:24:18.230357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453c6c0-9ecf-47f5-a177-62080011e058 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:24:18.249631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453c6c0-9ecf-47f5-a177-62080011e058 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:24:18.256667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6aef9c-e4fe-404d-909e-ffb75a437e0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:24:18.258260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6aef9c-e4fe-404d-909e-ffb75a437e0e service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:18.258683 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6aef9c-e4fe-404d-909e-ffb75a437e0e service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:18.259036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6aef9c-e4fe-404d-909e-ffb75a437e0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:18.269199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6aef9c-e4fe-404d-909e-ffb75a437e0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:24:18.274260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6aef9c-e4fe-404d-909e-ffb75a437e0e service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:24:18.285102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 770/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:24:18.694401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c6ccc-d502-4346-8bb9-62276002536d None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:18.695279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c6ccc-d502-4346-8bb9-62276002536d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:18.695547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c6ccc-d502-4346-8bb9-62276002536d None None] PATH_INFO: `/v3` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:18.697030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 747/1517] 10.222.0.30 () {54 vars in 925 bytes} [Mon Aug 22 17:24:18 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:24:18.704219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d00b0-1016-48e1-9cc1-950da8d72155 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:18.704553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d00b0-1016-48e1-9cc1-950da8d72155 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:18.704871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d00b0-1016-48e1-9cc1-950da8d72155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:18.774589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-168d00b0-1016-48e1-9cc1-950da8d72155 None None] MFA Rules not processed for user `26b5021285c44db9b05f9d15ceaf797f`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:24:18.776645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-168d00b0-1016-48e1-9cc1-950da8d72155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:18.902985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 771/1518] 10.222.0.30 () {58 vars in 1015 bytes} [Mon Aug 22 17:24:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:24:23.379264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070fa5-c5e6-4610-9386-d87d468de5c1 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:23.379904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070fa5-c5e6-4610-9386-d87d468de5c1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:23.379904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070fa5-c5e6-4610-9386-d87d468de5c1 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:23.381221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 748/1519] 10.222.0.20 () {54 vars in 961 bytes} [Mon Aug 22 17:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:24:23.391545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b04f7f-f31a-4b3e-b90d-c18e477c17a2 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:23.392425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b04f7f-f31a-4b3e-b90d-c18e477c17a2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:23.392425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b04f7f-f31a-4b3e-b90d-c18e477c17a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:23.415534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-23b04f7f-f31a-4b3e-b90d-c18e477c17a2 None None] MFA Rules not processed for user `26b5021285c44db9b05f9d15ceaf797f`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:24:23.416883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-23b04f7f-f31a-4b3e-b90d-c18e477c17a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:23.426115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 772/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Aug 22 17:24:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:24:23.437896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e070fa5-c5e6-4610-9386-d87d468de5c1 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:24:23.453029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e070fa5-c5e6-4610-9386-d87d468de5c1 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:24:23.458453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd58f2-2d1d-40b6-b720-88769ac7a91e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:24:23.459113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd58f2-2d1d-40b6-b720-88769ac7a91e service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:23.459391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd58f2-2d1d-40b6-b720-88769ac7a91e service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:23.459662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd58f2-2d1d-40b6-b720-88769ac7a91e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:23.470306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd58f2-2d1d-40b6-b720-88769ac7a91e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:24:23.472072 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd58f2-2d1d-40b6-b720-88769ac7a91e service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:24:23.484159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 749/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:24:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:24:29.619500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aec60-5eaa-4a7b-aa62-6d6a1cfb0f91 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:29.620125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aec60-5eaa-4a7b-aa62-6d6a1cfb0f91 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:29.620228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aec60-5eaa-4a7b-aa62-6d6a1cfb0f91 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:29.622789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:24:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:24:29.629734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c4e9-6c95-4521-aeeb-c9768c7d2807 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:29.630277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c4e9-6c95-4521-aeeb-c9768c7d2807 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:29.630763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c4e9-6c95-4521-aeeb-c9768c7d2807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:29.654544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-70a5c4e9-6c95-4521-aeeb-c9768c7d2807 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:24:29.655812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-70a5c4e9-6c95-4521-aeeb-c9768c7d2807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:29.664669 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:24:29.828672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667aec60-5eaa-4a7b-aa62-6d6a1cfb0f91 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:24:29.856765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667aec60-5eaa-4a7b-aa62-6d6a1cfb0f91 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:24:29.867166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd74e0-0550-4bce-9fe4-302780d16685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:24:29.870662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd74e0-0550-4bce-9fe4-302780d16685 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:29.870662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd74e0-0550-4bce-9fe4-302780d16685 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:29.870662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd74e0-0550-4bce-9fe4-302780d16685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:29.883822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd74e0-0550-4bce-9fe4-302780d16685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:24:29.885614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd74e0-0550-4bce-9fe4-302780d16685 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:24:29.896800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 774/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:24:30.786721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34adcfd-efc1-4d4d-a6ce-a014302b118b None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:30.787366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34adcfd-efc1-4d4d-a6ce-a014302b118b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:30.787366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34adcfd-efc1-4d4d-a6ce-a014302b118b None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:30.788994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 17:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:24:30.795093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6ddab-0ddd-444d-96a8-79f0181fb5f1 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:30.795964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6ddab-0ddd-444d-96a8-79f0181fb5f1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:30.795964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6ddab-0ddd-444d-96a8-79f0181fb5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:30.817698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e4f6ddab-0ddd-444d-96a8-79f0181fb5f1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:24:30.819109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e4f6ddab-0ddd-444d-96a8-79f0181fb5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:30.828841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 17:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:24:30.991578 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34adcfd-efc1-4d4d-a6ce-a014302b118b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:24:31.007519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34adcfd-efc1-4d4d-a6ce-a014302b118b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:24:31.013534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83bc0bc-8fa8-4f57-b49e-18e6eeb3b8f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:24:31.014246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bc0bc-8fa8-4f57-b49e-18e6eeb3b8f4 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:31.014616 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bc0bc-8fa8-4f57-b49e-18e6eeb3b8f4 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:31.015046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bc0bc-8fa8-4f57-b49e-18e6eeb3b8f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:31.025518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83bc0bc-8fa8-4f57-b49e-18e6eeb3b8f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:24:31.027330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83bc0bc-8fa8-4f57-b49e-18e6eeb3b8f4 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:24:31.040083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 752/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:24:35.339276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a62fd0-deb4-423d-9ff3-5eb057ccd46c None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:35.339987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a62fd0-deb4-423d-9ff3-5eb057ccd46c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:35.339987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a62fd0-deb4-423d-9ff3-5eb057ccd46c None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:35.341363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:24:35.347608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84bff9-dc62-4815-9358-0ce0039db544 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:35.348254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84bff9-dc62-4815-9358-0ce0039db544 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:35.348254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84bff9-dc62-4815-9358-0ce0039db544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:35.370763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ba84bff9-dc62-4815-9358-0ce0039db544 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:24:35.372069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ba84bff9-dc62-4815-9358-0ce0039db544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:35.381073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:24:35.689125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a62fd0-deb4-423d-9ff3-5eb057ccd46c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:24:35.691078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b6a62fd0-deb4-423d-9ff3-5eb057ccd46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:35.723922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:24:35.760654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a62fd0-deb4-423d-9ff3-5eb057ccd46c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:24:35.765829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cf9ad-2be2-412c-a1c4-4a37cec03d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:24:35.766465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf9ad-2be2-412c-a1c4-4a37cec03d01 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:35.766768 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf9ad-2be2-412c-a1c4-4a37cec03d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:35.767037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf9ad-2be2-412c-a1c4-4a37cec03d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:35.779009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cf9ad-2be2-412c-a1c4-4a37cec03d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:24:35.780493 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cf9ad-2be2-412c-a1c4-4a37cec03d01 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:24:35.792199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 777/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:24:37.831573 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e0df9-c8d0-4faf-818c-59ce5ba816db None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:37.832166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e0df9-c8d0-4faf-818c-59ce5ba816db None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:37.832318 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e0df9-c8d0-4faf-818c-59ce5ba816db None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:37.833894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:24:37.840235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00e927-f122-4e11-aee0-25c933b38398 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:37.841075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00e927-f122-4e11-aee0-25c933b38398 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:37.841075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00e927-f122-4e11-aee0-25c933b38398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:37.862816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8b00e927-f122-4e11-aee0-25c933b38398 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:24:37.864068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8b00e927-f122-4e11-aee0-25c933b38398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:37.873326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:24:38.176657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e0df9-c8d0-4faf-818c-59ce5ba816db None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:24:38.187703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=68 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:24:38.198880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e0df9-c8d0-4faf-818c-59ce5ba816db None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:24:38.205096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f9bd-655d-4298-b37d-d278658880cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:24:38.206167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f9bd-655d-4298-b37d-d278658880cf service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:38.206701 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f9bd-655d-4298-b37d-d278658880cf service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:38.207191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f9bd-655d-4298-b37d-d278658880cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:38.224924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08f9bd-655d-4298-b37d-d278658880cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:24:38.228491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08f9bd-655d-4298-b37d-d278658880cf service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:24:38.238889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 755/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:24:41.919741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f4498-e743-45cf-bd0f-71a45409c4e8 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:41.920380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f4498-e743-45cf-bd0f-71a45409c4e8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:41.920380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f4498-e743-45cf-bd0f-71a45409c4e8 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:41.921706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:24:41.927634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b395a2-867d-4cf2-b480-b959419055c4 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:41.928289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b395a2-867d-4cf2-b480-b959419055c4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:41.928289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b395a2-867d-4cf2-b480-b959419055c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:41.950083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b8b395a2-867d-4cf2-b480-b959419055c4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:24:41.951367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b8b395a2-867d-4cf2-b480-b959419055c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:41.959801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:24:42.540514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f4498-e743-45cf-bd0f-71a45409c4e8 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:24:42.542355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d38f4498-e743-45cf-bd0f-71a45409c4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:42.611804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f4498-e743-45cf-bd0f-71a45409c4e8 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:24:42.616407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421b0e7-55ea-4835-944d-b0cb9bc1552d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:24:42.617246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421b0e7-55ea-4835-944d-b0cb9bc1552d service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:42.617737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421b0e7-55ea-4835-944d-b0cb9bc1552d service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:42.618229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421b0e7-55ea-4835-944d-b0cb9bc1552d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:42.630594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421b0e7-55ea-4835-944d-b0cb9bc1552d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:24:42.632720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421b0e7-55ea-4835-944d-b0cb9bc1552d service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:24:42.644180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 780/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:24:44.813228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98e304-be9c-4a25-82ee-dc17c610338c None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:44.813844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98e304-be9c-4a25-82ee-dc17c610338c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:44.813844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98e304-be9c-4a25-82ee-dc17c610338c None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:44.815248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 17:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:24:44.821933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e4e9d-d7fb-4983-accc-347a018d4e2c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:44.822492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e4e9d-d7fb-4983-accc-347a018d4e2c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:44.822492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e4e9d-d7fb-4983-accc-347a018d4e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:44.844433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-573e4e9d-d7fb-4983-accc-347a018d4e2c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:24:44.845694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-573e4e9d-d7fb-4983-accc-347a018d4e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:44.855326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 17:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:24:45.439859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98e304-be9c-4a25-82ee-dc17c610338c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:24:45.441953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3f98e304-be9c-4a25-82ee-dc17c610338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:24:45.457387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98e304-be9c-4a25-82ee-dc17c610338c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:24:45.461535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33990434-388b-4813-b6ef-83a732709885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:24:45.462181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990434-388b-4813-b6ef-83a732709885 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:24:45.462455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990434-388b-4813-b6ef-83a732709885 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:24:45.462757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990434-388b-4813-b6ef-83a732709885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:24:45.472957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33990434-388b-4813-b6ef-83a732709885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:24:45.474446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33990434-388b-4813-b6ef-83a732709885 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:24:45.485729 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 758/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:24:54.838194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=47 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:24:55.480789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=14 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:18.720549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bee25-41e2-4e13-a8ac-6f10abdd537c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:18.721211 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bee25-41e2-4e13-a8ac-6f10abdd537c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:18.721496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bee25-41e2-4e13-a8ac-6f10abdd537c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:18.733391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee4248-dd40-4d7f-b0c3-ac30cb7ba077 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:18.734022 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee4248-dd40-4d7f-b0c3-ac30cb7ba077 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:18.734376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee4248-dd40-4d7f-b0c3-ac30cb7ba077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:18.750924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-157bee25-41e2-4e13-a8ac-6f10abdd537c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:18.752339 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-157bee25-41e2-4e13-a8ac-6f10abdd537c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:18.771125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 782/1540] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:18.771125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e3d00-5064-459e-9f47-4cda27fd5148 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:18.771125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e3d00-5064-459e-9f47-4cda27fd5148 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:18.771125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e3d00-5064-459e-9f47-4cda27fd5148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:18.796846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3cee4248-dd40-4d7f-b0c3-ac30cb7ba077 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:18.802245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3cee4248-dd40-4d7f-b0c3-ac30cb7ba077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:18.812202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-719e3d00-5064-459e-9f47-4cda27fd5148 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:18.813501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-719e3d00-5064-459e-9f47-4cda27fd5148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:18.830694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 783/1541] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:18.832503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6ab74-c742-47e3-bdab-069e7c47df31 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:18.832793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6ab74-c742-47e3-bdab-069e7c47df31 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:18.833073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6ab74-c742-47e3-bdab-069e7c47df31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:18.849505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 759/1542] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:18.852851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18f0d5-ec90-40fd-a46b-5fea3d38e83f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:18.853345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18f0d5-ec90-40fd-a46b-5fea3d38e83f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:18.853820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18f0d5-ec90-40fd-a46b-5fea3d38e83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:18.876148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e7f6ab74-c742-47e3-bdab-069e7c47df31 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:18.877430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e7f6ab74-c742-47e3-bdab-069e7c47df31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:18.908373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 784/1543] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:18.910447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9df2f9-b4c8-4fdb-83f2-78bf4da6b2e4 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:18.910763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9df2f9-b4c8-4fdb-83f2-78bf4da6b2e4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:18.911039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9df2f9-b4c8-4fdb-83f2-78bf4da6b2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:18.940917 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-8e18f0d5-ec90-40fd-a46b-5fea3d38e83f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:18.943703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8e18f0d5-ec90-40fd-a46b-5fea3d38e83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:18.960431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2c9df2f9-b4c8-4fdb-83f2-78bf4da6b2e4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:18.961802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2c9df2f9-b4c8-4fdb-83f2-78bf4da6b2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:18.971209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 785/1544] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:18.972771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1b63a-139a-4ee9-baf5-aee4553c7050 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:18.973004 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1b63a-139a-4ee9-baf5-aee4553c7050 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:18.973279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1b63a-139a-4ee9-baf5-aee4553c7050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.000655 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 760/1545] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:19.003007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3c844-4cae-49e5-b361-a68bf0a9e388 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.003491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3c844-4cae-49e5-b361-a68bf0a9e388 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.003982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3c844-4cae-49e5-b361-a68bf0a9e388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.023310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-bbf1b63a-139a-4ee9-baf5-aee4553c7050 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:19.025208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bbf1b63a-139a-4ee9-baf5-aee4553c7050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:19.058425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 786/1546] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:19.060755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec08bf-e1d1-450f-8ad4-46e180bf921b None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.061147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec08bf-e1d1-450f-8ad4-46e180bf921b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.061602 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec08bf-e1d1-450f-8ad4-46e180bf921b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.087875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-f5c3c844-4cae-49e5-b361-a68bf0a9e388 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:19.091134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f5c3c844-4cae-49e5-b361-a68bf0a9e388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:19.126642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 761/1547] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:19.128343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3c844-4cae-49e5-b361-a68bf0a9e388 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:19.141316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-faec08bf-e1d1-450f-8ad4-46e180bf921b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:19.148564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-faec08bf-e1d1-450f-8ad4-46e180bf921b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:19.169185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3c844-4cae-49e5-b361-a68bf0a9e388 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:19.174196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 787/1548] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:19.175130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec08bf-e1d1-450f-8ad4-46e180bf921b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:19.198254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec08bf-e1d1-450f-8ad4-46e180bf921b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:19.200245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05babd-fd9d-4f45-b06e-5331585e8ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:19.201589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05babd-fd9d-4f45-b06e-5331585e8ebc admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.202341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05babd-fd9d-4f45-b06e-5331585e8ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.210268 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05babd-fd9d-4f45-b06e-5331585e8ebc admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.211864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05babd-fd9d-4f45-b06e-5331585e8ebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:19.212450 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05babd-fd9d-4f45-b06e-5331585e8ebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:19.214347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: WARNING py.warnings [None req-4e05babd-fd9d-4f45-b06e-5331585e8ebc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:26:19.214347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: warnings.warn(msg) Aug 22 17:26:19.214347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: Aug 22 17:26:19.215611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a274c8-7199-475a-bd26-eb78f7331d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:19.216054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05babd-fd9d-4f45-b06e-5331585e8ebc admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:19.217148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a274c8-7199-475a-bd26-eb78f7331d62 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.217880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a274c8-7199-475a-bd26-eb78f7331d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.218550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a274c8-7199-475a-bd26-eb78f7331d62 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.219944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a274c8-7199-475a-bd26-eb78f7331d62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:19.220766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a274c8-7199-475a-bd26-eb78f7331d62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:19.223604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a274c8-7199-475a-bd26-eb78f7331d62 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:19.242274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 762/1549] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:19.243462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05babd-fd9d-4f45-b06e-5331585e8ebc admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:19.249754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 788/1550] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:19.251735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a274c8-7199-475a-bd26-eb78f7331d62 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:19.263498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05babd-fd9d-4f45-b06e-5331585e8ebc admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:19.273036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c191df0-008d-475d-9d68-c21b893d7261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:19.274139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c191df0-008d-475d-9d68-c21b893d7261 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.274567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c191df0-008d-475d-9d68-c21b893d7261 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.275048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c191df0-008d-475d-9d68-c21b893d7261 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.276462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c191df0-008d-475d-9d68-c21b893d7261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:19.277115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c191df0-008d-475d-9d68-c21b893d7261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:19.277986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a274c8-7199-475a-bd26-eb78f7331d62 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:19.280317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c191df0-008d-475d-9d68-c21b893d7261 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:19.289535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f02fc-6b1b-4278-ba1f-6a0b2928f88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:19.291334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f02fc-6b1b-4278-ba1f-6a0b2928f88e admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.293153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f02fc-6b1b-4278-ba1f-6a0b2928f88e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.294146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f02fc-6b1b-4278-ba1f-6a0b2928f88e admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.295893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f02fc-6b1b-4278-ba1f-6a0b2928f88e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:19.296681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f02fc-6b1b-4278-ba1f-6a0b2928f88e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:19.301485 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f02fc-6b1b-4278-ba1f-6a0b2928f88e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:19.309581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 763/1551] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:19.311988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a4939-d098-41f4-a7e6-9f083c70fa1e None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.312533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a4939-d098-41f4-a7e6-9f083c70fa1e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.312997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a4939-d098-41f4-a7e6-9f083c70fa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.338529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 789/1552] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:19.339908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02363c24-169f-4dfb-9cd6-1c78b965f583 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.340200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02363c24-169f-4dfb-9cd6-1c78b965f583 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.340475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02363c24-169f-4dfb-9cd6-1c78b965f583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.352467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b41a4939-d098-41f4-a7e6-9f083c70fa1e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:19.354162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b41a4939-d098-41f4-a7e6-9f083c70fa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:19.369066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 764/1553] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:19.369066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a4939-d098-41f4-a7e6-9f083c70fa1e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:19.369757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-02363c24-169f-4dfb-9cd6-1c78b965f583 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:19.371089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-02363c24-169f-4dfb-9cd6-1c78b965f583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:19.384728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 790/1554] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:19.384728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02363c24-169f-4dfb-9cd6-1c78b965f583 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:19.385211 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a4939-d098-41f4-a7e6-9f083c70fa1e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:19.393433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e207b-da39-4e3e-a48f-ed92ef7eef6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:19.394375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02363c24-169f-4dfb-9cd6-1c78b965f583 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:19.394810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e207b-da39-4e3e-a48f-ed92ef7eef6b admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.395524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e207b-da39-4e3e-a48f-ed92ef7eef6b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.396080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e207b-da39-4e3e-a48f-ed92ef7eef6b admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.398105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e207b-da39-4e3e-a48f-ed92ef7eef6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:19.400544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d27ac-1a79-4ff7-8ae3-65f8ba3e3127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:19.401262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d27ac-1a79-4ff7-8ae3-65f8ba3e3127 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.401607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d27ac-1a79-4ff7-8ae3-65f8ba3e3127 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.401925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e207b-da39-4e3e-a48f-ed92ef7eef6b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:19.402112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d27ac-1a79-4ff7-8ae3-65f8ba3e3127 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.402762 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d27ac-1a79-4ff7-8ae3-65f8ba3e3127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:19.403876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WARNING py.warnings [None req-9d5d27ac-1a79-4ff7-8ae3-65f8ba3e3127 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:26:19.403876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: warnings.warn(msg) Aug 22 17:26:19.403876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: Aug 22 17:26:19.405610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d27ac-1a79-4ff7-8ae3-65f8ba3e3127 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:19.541894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 765/1555] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:19.542685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e207b-da39-4e3e-a48f-ed92ef7eef6b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:19.558396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 791/1556] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:19.558396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25f9aa-2223-4ff0-b9e2-23ceff315dbf None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.558396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25f9aa-2223-4ff0-b9e2-23ceff315dbf None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.558396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25f9aa-2223-4ff0-b9e2-23ceff315dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.581072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8c25f9aa-2223-4ff0-b9e2-23ceff315dbf None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:19.582333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8c25f9aa-2223-4ff0-b9e2-23ceff315dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:19.652256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e207b-da39-4e3e-a48f-ed92ef7eef6b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:19.673549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 792/1557] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:19.674749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25f9aa-2223-4ff0-b9e2-23ceff315dbf None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:19.689916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25f9aa-2223-4ff0-b9e2-23ceff315dbf None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:19.698784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56ff4f-5567-42a0-91a2-d31b0255488c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:19.699467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56ff4f-5567-42a0-91a2-d31b0255488c admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.699752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56ff4f-5567-42a0-91a2-d31b0255488c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.700026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56ff4f-5567-42a0-91a2-d31b0255488c admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.700679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56ff4f-5567-42a0-91a2-d31b0255488c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:19.702808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56ff4f-5567-42a0-91a2-d31b0255488c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:19.747119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a565334-c7aa-47b6-82ec-3bcd696ea14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:19.748150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a565334-c7aa-47b6-82ec-3bcd696ea14e admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.748634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a565334-c7aa-47b6-82ec-3bcd696ea14e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.749108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a565334-c7aa-47b6-82ec-3bcd696ea14e admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.750358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a565334-c7aa-47b6-82ec-3bcd696ea14e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:19.758029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a565334-c7aa-47b6-82ec-3bcd696ea14e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:19.806829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 793/1558] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:19.806829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56ff4f-5567-42a0-91a2-d31b0255488c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:19.864821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:19.885496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56ff4f-5567-42a0-91a2-d31b0255488c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:19.891260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68140e6-1fb6-42ac-9773-dc9be738bb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:19.892111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68140e6-1fb6-42ac-9773-dc9be738bb18 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.892516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68140e6-1fb6-42ac-9773-dc9be738bb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.892878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68140e6-1fb6-42ac-9773-dc9be738bb18 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.893683 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68140e6-1fb6-42ac-9773-dc9be738bb18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:19.894046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68140e6-1fb6-42ac-9773-dc9be738bb18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:19.895953 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68140e6-1fb6-42ac-9773-dc9be738bb18 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:19.910748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 794/1559] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:19.912226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5c654-59e0-4246-9a75-915983087074 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.912741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5c654-59e0-4246-9a75-915983087074 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.913138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5c654-59e0-4246-9a75-915983087074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.930438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 766/1560] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/projects => generated 367 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:19.933390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62459a-cbd0-48d1-9759-bab54599aad7 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:19.934254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62459a-cbd0-48d1-9759-bab54599aad7 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:19.935026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62459a-cbd0-48d1-9759-bab54599aad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:19.938786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0ae5c654-59e0-4246-9a75-915983087074 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:19.940039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0ae5c654-59e0-4246-9a75-915983087074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:19.968701 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-fb62459a-cbd0-48d1-9759-bab54599aad7 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:19.970745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fb62459a-cbd0-48d1-9759-bab54599aad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:19.995538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:20.023762 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 795/1561] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:20.024692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5c654-59e0-4246-9a75-915983087074 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.035268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5c654-59e0-4246-9a75-915983087074 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.041956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde9cc2-f670-4d8e-b768-e79bc4404038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.042668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde9cc2-f670-4d8e-b768-e79bc4404038 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.042950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde9cc2-f670-4d8e-b768-e79bc4404038 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.043248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde9cc2-f670-4d8e-b768-e79bc4404038 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.043904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde9cc2-f670-4d8e-b768-e79bc4404038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.046254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde9cc2-f670-4d8e-b768-e79bc4404038 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.089095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 767/1562] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:20.090351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62459a-cbd0-48d1-9759-bab54599aad7 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.108864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62459a-cbd0-48d1-9759-bab54599aad7 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.118635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb77b3-4339-4d0a-9c7d-775f11799fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.119675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb77b3-4339-4d0a-9c7d-775f11799fae admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.120148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb77b3-4339-4d0a-9c7d-775f11799fae admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.120644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb77b3-4339-4d0a-9c7d-775f11799fae admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.121769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb77b3-4339-4d0a-9c7d-775f11799fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.123728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: WARNING py.warnings [None req-89fb77b3-4339-4d0a-9c7d-775f11799fae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:26:20.123728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: warnings.warn(msg) Aug 22 17:26:20.123728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: Aug 22 17:26:20.126157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb77b3-4339-4d0a-9c7d-775f11799fae admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.147169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 796/1563] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:20 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:20.148929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171c04d-8aa6-4a61-a00a-83693b2110a1 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.149290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171c04d-8aa6-4a61-a00a-83693b2110a1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.149636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171c04d-8aa6-4a61-a00a-83693b2110a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.173456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e171c04d-8aa6-4a61-a00a-83693b2110a1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:20.174767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e171c04d-8aa6-4a61-a00a-83693b2110a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:20.184194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 797/1564] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:20.185106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171c04d-8aa6-4a61-a00a-83693b2110a1 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.195989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171c04d-8aa6-4a61-a00a-83693b2110a1 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.201224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5238b-1a50-42c5-8ece-bd4a21e7eb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.201864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5238b-1a50-42c5-8ece-bd4a21e7eb84 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.202139 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5238b-1a50-42c5-8ece-bd4a21e7eb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.202409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5238b-1a50-42c5-8ece-bd4a21e7eb84 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.203098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5238b-1a50-42c5-8ece-bd4a21e7eb84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.205097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5238b-1a50-42c5-8ece-bd4a21e7eb84 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.238052 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 768/1565] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:20 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:20.239508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb77b3-4339-4d0a-9c7d-775f11799fae admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.253363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb77b3-4339-4d0a-9c7d-775f11799fae admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.260213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa9e62-cb50-4e8d-85a7-9b79ca5f7fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.260942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa9e62-cb50-4e8d-85a7-9b79ca5f7fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.261278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa9e62-cb50-4e8d-85a7-9b79ca5f7fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.261610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa9e62-cb50-4e8d-85a7-9b79ca5f7fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.262363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa9e62-cb50-4e8d-85a7-9b79ca5f7fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.265242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa9e62-cb50-4e8d-85a7-9b79ca5f7fb1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.307359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 798/1566] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:20 2022] POST /identity/v3/users => generated 283 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:20.308648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5238b-1a50-42c5-8ece-bd4a21e7eb84 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.322000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5238b-1a50-42c5-8ece-bd4a21e7eb84 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.329978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605673b4-0f14-48c0-a6e0-073b825a2c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.331087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605673b4-0f14-48c0-a6e0-073b825a2c42 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.331562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605673b4-0f14-48c0-a6e0-073b825a2c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.332018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605673b4-0f14-48c0-a6e0-073b825a2c42 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.333147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605673b4-0f14-48c0-a6e0-073b825a2c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.336935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605673b4-0f14-48c0-a6e0-073b825a2c42 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.379200 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 769/1567] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:20 2022] POST /identity/v3/projects => generated 393 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:20.380502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa9e62-cb50-4e8d-85a7-9b79ca5f7fb1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.439964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 799/1568] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:20 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:20.439964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638b4a-3a4d-40ae-afa2-4456f92c710d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.439964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638b4a-3a4d-40ae-afa2-4456f92c710d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.439964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638b4a-3a4d-40ae-afa2-4456f92c710d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.468264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-48638b4a-3a4d-40ae-afa2-4456f92c710d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:20.469897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-48638b4a-3a4d-40ae-afa2-4456f92c710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:20.481756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 800/1569] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:20.481756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48638b4a-3a4d-40ae-afa2-4456f92c710d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.486895 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa9e62-cb50-4e8d-85a7-9b79ca5f7fb1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.493770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b65d85-4494-4900-bb9f-37ccec17fcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.495107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b65d85-4494-4900-bb9f-37ccec17fcba admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.495456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b65d85-4494-4900-bb9f-37ccec17fcba admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.495863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b65d85-4494-4900-bb9f-37ccec17fcba admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.496810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b65d85-4494-4900-bb9f-37ccec17fcba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:20.497403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b65d85-4494-4900-bb9f-37ccec17fcba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.499209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b65d85-4494-4900-bb9f-37ccec17fcba admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.502465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48638b4a-3a4d-40ae-afa2-4456f92c710d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.507782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033678-b836-41d0-9b5f-5a8a68c644cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.508725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033678-b836-41d0-9b5f-5a8a68c644cb admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.509024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033678-b836-41d0-9b5f-5a8a68c644cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.509297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033678-b836-41d0-9b5f-5a8a68c644cb admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.509991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29033678-b836-41d0-9b5f-5a8a68c644cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.512103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29033678-b836-41d0-9b5f-5a8a68c644cb admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.533305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 770/1570] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:20.533305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b65d85-4494-4900-bb9f-37ccec17fcba admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.533754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 801/1571] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:20.533754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033678-b836-41d0-9b5f-5a8a68c644cb admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.541559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b65d85-4494-4900-bb9f-37ccec17fcba admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.541896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033678-b836-41d0-9b5f-5a8a68c644cb admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.548655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e60963-2dca-44d9-a00f-b58b1bf15fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.549288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e60963-2dca-44d9-a00f-b58b1bf15fef admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.549487 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f852bfb-229f-411c-b03c-0193ccfe7eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.549668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e60963-2dca-44d9-a00f-b58b1bf15fef admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.549838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e60963-2dca-44d9-a00f-b58b1bf15fef admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.550353 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f852bfb-229f-411c-b03c-0193ccfe7eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.550551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e60963-2dca-44d9-a00f-b58b1bf15fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.550834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f852bfb-229f-411c-b03c-0193ccfe7eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.551201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f852bfb-229f-411c-b03c-0193ccfe7eb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.552009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f852bfb-229f-411c-b03c-0193ccfe7eb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:20.552184 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e60963-2dca-44d9-a00f-b58b1bf15fef admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.552471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f852bfb-229f-411c-b03c-0193ccfe7eb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.555828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f852bfb-229f-411c-b03c-0193ccfe7eb6 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.565496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 802/1572] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:20.566586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e60963-2dca-44d9-a00f-b58b1bf15fef admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.575136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 771/1573] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:20.576220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f852bfb-229f-411c-b03c-0193ccfe7eb6 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.578605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e60963-2dca-44d9-a00f-b58b1bf15fef admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.583946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebaa9f3-b089-4cc9-92d8-8c431802ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.586279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebaa9f3-b089-4cc9-92d8-8c431802ec45 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.586279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebaa9f3-b089-4cc9-92d8-8c431802ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.586279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebaa9f3-b089-4cc9-92d8-8c431802ec45 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.586661 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebaa9f3-b089-4cc9-92d8-8c431802ec45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.588348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebaa9f3-b089-4cc9-92d8-8c431802ec45 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.592404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f852bfb-229f-411c-b03c-0193ccfe7eb6 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.599234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d0990-2163-4aef-957c-2369d7c02db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.600462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d0990-2163-4aef-957c-2369d7c02db1 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.600944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d0990-2163-4aef-957c-2369d7c02db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.601417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d0990-2163-4aef-957c-2369d7c02db1 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.602370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d0990-2163-4aef-957c-2369d7c02db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.604818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d0990-2163-4aef-957c-2369d7c02db1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.606205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 803/1574] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:20.607539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebaa9f3-b089-4cc9-92d8-8c431802ec45 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.628673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebaa9f3-b089-4cc9-92d8-8c431802ec45 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.639559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a06ee9-5b36-4084-a2fc-ea26552bbb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.639559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a06ee9-5b36-4084-a2fc-ea26552bbb14 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.640976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a06ee9-5b36-4084-a2fc-ea26552bbb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.641392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a06ee9-5b36-4084-a2fc-ea26552bbb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.642626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a06ee9-5b36-4084-a2fc-ea26552bbb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.645751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a06ee9-5b36-4084-a2fc-ea26552bbb14 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.669435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 804/1575] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:20.671019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a06ee9-5b36-4084-a2fc-ea26552bbb14 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.690506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a06ee9-5b36-4084-a2fc-ea26552bbb14 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.702923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af026290-f995-4c0f-9776-75eeb5023453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.702923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af026290-f995-4c0f-9776-75eeb5023453 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.702923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af026290-f995-4c0f-9776-75eeb5023453 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.702923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af026290-f995-4c0f-9776-75eeb5023453 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.703524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af026290-f995-4c0f-9776-75eeb5023453 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:20.703524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af026290-f995-4c0f-9776-75eeb5023453 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.706280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af026290-f995-4c0f-9776-75eeb5023453 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.731784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 805/1576] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:20.733275 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af026290-f995-4c0f-9776-75eeb5023453 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.754276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af026290-f995-4c0f-9776-75eeb5023453 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.761584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94c888-3bcd-49b4-a61a-4bbdb36c90b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.762440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94c888-3bcd-49b4-a61a-4bbdb36c90b7 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.762879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94c888-3bcd-49b4-a61a-4bbdb36c90b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.763295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94c888-3bcd-49b4-a61a-4bbdb36c90b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.764022 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94c888-3bcd-49b4-a61a-4bbdb36c90b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.765970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94c888-3bcd-49b4-a61a-4bbdb36c90b7 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.779686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 806/1577] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:20.781340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94c888-3bcd-49b4-a61a-4bbdb36c90b7 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.790989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:20.795711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94c888-3bcd-49b4-a61a-4bbdb36c90b7 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.802319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6ca05-7e5c-408f-bf5a-1bb78b4a5e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.803076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6ca05-7e5c-408f-bf5a-1bb78b4a5e4e admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.803388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6ca05-7e5c-408f-bf5a-1bb78b4a5e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.803676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6ca05-7e5c-408f-bf5a-1bb78b4a5e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.804606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6ca05-7e5c-408f-bf5a-1bb78b4a5e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.807376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6ca05-7e5c-408f-bf5a-1bb78b4a5e4e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.921445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 772/1578] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:20 2022] POST /identity/v3/users => generated 297 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:20.922639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d0990-2163-4aef-957c-2369d7c02db1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.935760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d0990-2163-4aef-957c-2369d7c02db1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.941168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc3e7c-1a86-4fae-9d85-7a2a94f88b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.941831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc3e7c-1a86-4fae-9d85-7a2a94f88b13 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.942109 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc3e7c-1a86-4fae-9d85-7a2a94f88b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.942379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc3e7c-1a86-4fae-9d85-7a2a94f88b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.943092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc3e7c-1a86-4fae-9d85-7a2a94f88b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.945880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc3e7c-1a86-4fae-9d85-7a2a94f88b13 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:20.958415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 773/1579] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:20.960360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc3e7c-1a86-4fae-9d85-7a2a94f88b13 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:20.972951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc3e7c-1a86-4fae-9d85-7a2a94f88b13 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:20.979025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f17a34-0c5b-4449-8fe0-81f0108ddd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:20.985812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f17a34-0c5b-4449-8fe0-81f0108ddd19 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:20.985812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f17a34-0c5b-4449-8fe0-81f0108ddd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:20.985812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f17a34-0c5b-4449-8fe0-81f0108ddd19 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:20.985812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f17a34-0c5b-4449-8fe0-81f0108ddd19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:20.985812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f17a34-0c5b-4449-8fe0-81f0108ddd19 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:21.018505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 807/1580] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:20 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:21.019532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6ca05-7e5c-408f-bf5a-1bb78b4a5e4e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:21.110532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6ca05-7e5c-408f-bf5a-1bb78b4a5e4e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:21.113155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 774/1581] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:20 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:21.114778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f17a34-0c5b-4449-8fe0-81f0108ddd19 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:21.132613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:21.199184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f17a34-0c5b-4449-8fe0-81f0108ddd19 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:21.204205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bfdbd-582a-4e84-b22b-37f2227c65f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:21.204875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bfdbd-582a-4e84-b22b-37f2227c65f1 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:21.205272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bfdbd-582a-4e84-b22b-37f2227c65f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:21.205677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bfdbd-582a-4e84-b22b-37f2227c65f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:21.206560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bfdbd-582a-4e84-b22b-37f2227c65f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:21.208117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a85a52-d362-49e4-ad8e-244112680c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:21.208382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bfdbd-582a-4e84-b22b-37f2227c65f1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:21.208972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a85a52-d362-49e4-ad8e-244112680c8a admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:21.209892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a85a52-d362-49e4-ad8e-244112680c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:21.211666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a85a52-d362-49e4-ad8e-244112680c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:21.212296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a85a52-d362-49e4-ad8e-244112680c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:21.214122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a85a52-d362-49e4-ad8e-244112680c8a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:21.220853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 775/1582] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:21.222432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bfdbd-582a-4e84-b22b-37f2227c65f1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:21.233178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 808/1583] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:21.233881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a85a52-d362-49e4-ad8e-244112680c8a admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:21.234392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bfdbd-582a-4e84-b22b-37f2227c65f1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:21.240349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc2c33-4c6e-4415-8df4-c4b9eb655b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:21.241802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc2c33-4c6e-4415-8df4-c4b9eb655b25 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:21.242083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc2c33-4c6e-4415-8df4-c4b9eb655b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:21.242358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc2c33-4c6e-4415-8df4-c4b9eb655b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:21.243217 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc2c33-4c6e-4415-8df4-c4b9eb655b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:21.245325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc2c33-4c6e-4415-8df4-c4b9eb655b25 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:21.246218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a85a52-d362-49e4-ad8e-244112680c8a admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:21.251596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83ee6a-68f3-4a87-a552-d88c2911bc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:21.252478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83ee6a-68f3-4a87-a552-d88c2911bc07 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:21.252756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83ee6a-68f3-4a87-a552-d88c2911bc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:21.266173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83ee6a-68f3-4a87-a552-d88c2911bc07 admin admin] PATH_INFO: `/v3/projects/6436020e938047408d1f979e23dd48da/users/b0db362075b04b2294a6268576a5bba3/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:21.274808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83ee6a-68f3-4a87-a552-d88c2911bc07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6436020e938047408d1f979e23dd48da, user_id=b0db362075b04b2294a6268576a5bba3, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:21.276707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WARNING py.warnings [None req-ed83ee6a-68f3-4a87-a552-d88c2911bc07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:26:21.276707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: warnings.warn(msg) Aug 22 17:26:21.276707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: Aug 22 17:26:21.280979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83ee6a-68f3-4a87-a552-d88c2911bc07 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:21.393395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 776/1584] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:21 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:21.395255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc2c33-4c6e-4415-8df4-c4b9eb655b25 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:21.464609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 809/1585] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:21 2022] PUT /identity/v3/projects/6436020e938047408d1f979e23dd48da/users/b0db362075b04b2294a6268576a5bba3/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:21.466097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83ee6a-68f3-4a87-a552-d88c2911bc07 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:21.497377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc2c33-4c6e-4415-8df4-c4b9eb655b25 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:21.562327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83ee6a-68f3-4a87-a552-d88c2911bc07 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:21.569893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5a73-ad58-4577-b6b6-0a5875e7b373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:21.571076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5a73-ad58-4577-b6b6-0a5875e7b373 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:21.571422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5a73-ad58-4577-b6b6-0a5875e7b373 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:21.571842 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5a73-ad58-4577-b6b6-0a5875e7b373 admin admin] PATH_INFO: `/v3/projects/64f81017c0474297ac64ce306d739071/users/dcaea48773d2426eacacdcab56c1669b/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:21.578469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:21.589176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4bb2f8-34e4-4f82-a85e-31c6a558286d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:21.590158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4bb2f8-34e4-4f82-a85e-31c6a558286d admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:21.590645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4bb2f8-34e4-4f82-a85e-31c6a558286d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:21.591088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4bb2f8-34e4-4f82-a85e-31c6a558286d admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:21.592117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4bb2f8-34e4-4f82-a85e-31c6a558286d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:21.594877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4bb2f8-34e4-4f82-a85e-31c6a558286d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:21.599211 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d5a73-ad58-4577-b6b6-0a5875e7b373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f81017c0474297ac64ce306d739071, user_id=dcaea48773d2426eacacdcab56c1669b, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:21.604739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d5a73-ad58-4577-b6b6-0a5875e7b373 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:21.617491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 777/1586] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:21.617491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4bb2f8-34e4-4f82-a85e-31c6a558286d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:21.628547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4bb2f8-34e4-4f82-a85e-31c6a558286d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:21.633953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e736d-123a-45e8-8ac6-6909446b8bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:21.634709 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e736d-123a-45e8-8ac6-6909446b8bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:21.634992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e736d-123a-45e8-8ac6-6909446b8bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:21.635264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e736d-123a-45e8-8ac6-6909446b8bf8 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:21.635921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e736d-123a-45e8-8ac6-6909446b8bf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:21.637932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e736d-123a-45e8-8ac6-6909446b8bf8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:21.758544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 810/1587] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:21 2022] PUT /identity/v3/projects/64f81017c0474297ac64ce306d739071/users/dcaea48773d2426eacacdcab56c1669b/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:21.761389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5a73-ad58-4577-b6b6-0a5875e7b373 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:21.781056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 778/1588] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:21 2022] POST /identity/v3/users => generated 294 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:21.781056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e736d-123a-45e8-8ac6-6909446b8bf8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:21.791422 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e736d-123a-45e8-8ac6-6909446b8bf8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:21.799856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:21.864729 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a887b6-6dc5-4ce5-ae6c-2a0420f86624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:21.865481 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a887b6-6dc5-4ce5-ae6c-2a0420f86624 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:21.866272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a887b6-6dc5-4ce5-ae6c-2a0420f86624 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:21.866687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a887b6-6dc5-4ce5-ae6c-2a0420f86624 admin admin] PATH_INFO: `/v3/projects/e6a0059a8ba14a1aaab08cccc86fe6c6/users/9e88f01e082a4ebbad83de0e02de82b8/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:21.868975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5a73-ad58-4577-b6b6-0a5875e7b373 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:21.875051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28930284-0e85-4e01-a247-75d1ad5e5518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:21.876166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28930284-0e85-4e01-a247-75d1ad5e5518 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:21.876872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28930284-0e85-4e01-a247-75d1ad5e5518 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:21.877195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28930284-0e85-4e01-a247-75d1ad5e5518 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:21.877889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28930284-0e85-4e01-a247-75d1ad5e5518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:21.880374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28930284-0e85-4e01-a247-75d1ad5e5518 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:21.887805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a887b6-6dc5-4ce5-ae6c-2a0420f86624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a0059a8ba14a1aaab08cccc86fe6c6, user_id=9e88f01e082a4ebbad83de0e02de82b8, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:21.891548 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a887b6-6dc5-4ce5-ae6c-2a0420f86624 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:21.997476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 811/1589] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:21 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:21.998949 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28930284-0e85-4e01-a247-75d1ad5e5518 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.010803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28930284-0e85-4e01-a247-75d1ad5e5518 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.016065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d9deb-fd25-4c76-b916-015a9966c478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.016809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d9deb-fd25-4c76-b916-015a9966c478 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.017093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d9deb-fd25-4c76-b916-015a9966c478 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.017407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d9deb-fd25-4c76-b916-015a9966c478 admin admin] PATH_INFO: `/v3/projects/198c51c799bf40e98617b64389fc175c/users/9f129b7e883d4bf6ad4a9c0a02768aa0/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.030743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 779/1590] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:21 2022] PUT /identity/v3/projects/e6a0059a8ba14a1aaab08cccc86fe6c6/users/9e88f01e082a4ebbad83de0e02de82b8/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:22.043408 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a887b6-6dc5-4ce5-ae6c-2a0420f86624 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.043564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d9deb-fd25-4c76-b916-015a9966c478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=198c51c799bf40e98617b64389fc175c, user_id=9f129b7e883d4bf6ad4a9c0a02768aa0, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.043564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d9deb-fd25-4c76-b916-015a9966c478 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.122780 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a887b6-6dc5-4ce5-ae6c-2a0420f86624 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.128023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cda715-10c7-4da1-a91c-ab81cee1e4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.128653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda715-10c7-4da1-a91c-ab81cee1e4b9 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.128933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda715-10c7-4da1-a91c-ab81cee1e4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.129203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda715-10c7-4da1-a91c-ab81cee1e4b9 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.129852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cda715-10c7-4da1-a91c-ab81cee1e4b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.132030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cda715-10c7-4da1-a91c-ab81cee1e4b9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.156609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 812/1591] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:21 2022] PUT /identity/v3/projects/198c51c799bf40e98617b64389fc175c/users/9f129b7e883d4bf6ad4a9c0a02768aa0/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:22.162258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0d561-038a-4707-a74c-e81d27a54255 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.162258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0d561-038a-4707-a74c-e81d27a54255 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.162258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0d561-038a-4707-a74c-e81d27a54255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.180982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-11f0d561-038a-4707-a74c-e81d27a54255 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:22.182284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-11f0d561-038a-4707-a74c-e81d27a54255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:22.254343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 780/1592] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:22 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:22.258166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cda715-10c7-4da1-a91c-ab81cee1e4b9 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.263897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 813/1593] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:22.266400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e33eb-2314-4cf3-928f-bbe86cee3f01 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.266998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e33eb-2314-4cf3-928f-bbe86cee3f01 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.267439 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e33eb-2314-4cf3-928f-bbe86cee3f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.277805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cda715-10c7-4da1-a91c-ab81cee1e4b9 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.285612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d286dd-307d-4f09-9cf4-bd72186d06a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.286439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d286dd-307d-4f09-9cf4-bd72186d06a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.286805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d286dd-307d-4f09-9cf4-bd72186d06a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.287120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d286dd-307d-4f09-9cf4-bd72186d06a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.288607 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d286dd-307d-4f09-9cf4-bd72186d06a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.290403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d286dd-307d-4f09-9cf4-bd72186d06a1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.302782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 781/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:22.304353 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d286dd-307d-4f09-9cf4-bd72186d06a1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.310684 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-7d2e33eb-2314-4cf3-928f-bbe86cee3f01 None None] MFA Rules not processed for user `dcaea48773d2426eacacdcab56c1669b`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:22.311904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7d2e33eb-2314-4cf3-928f-bbe86cee3f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:22.323341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d286dd-307d-4f09-9cf4-bd72186d06a1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.329731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5259d-5e06-4452-821e-54db5f9e1443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.330855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5259d-5e06-4452-821e-54db5f9e1443 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.331400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5259d-5e06-4452-821e-54db5f9e1443 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.331760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5259d-5e06-4452-821e-54db5f9e1443 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.332836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5259d-5e06-4452-821e-54db5f9e1443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.335423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5259d-5e06-4452-821e-54db5f9e1443 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.358698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 782/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:22.358698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5259d-5e06-4452-821e-54db5f9e1443 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.370344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5259d-5e06-4452-821e-54db5f9e1443 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.375148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:22.375684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439edffe-88d2-41f1-b399-05e4881574b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.377488 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439edffe-88d2-41f1-b399-05e4881574b1 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.377488 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439edffe-88d2-41f1-b399-05e4881574b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.377841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439edffe-88d2-41f1-b399-05e4881574b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.379410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439edffe-88d2-41f1-b399-05e4881574b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.382363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439edffe-88d2-41f1-b399-05e4881574b1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.401853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 783/1596] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:22.401853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8a5fc-4739-40b7-9064-a9bfe4b3a5ac None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.401853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8a5fc-4739-40b7-9064-a9bfe4b3a5ac None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.401853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8a5fc-4739-40b7-9064-a9bfe4b3a5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.406155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 814/1597] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:22.411103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc904c6-c613-4612-bad8-deb4143ad8c0 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.411103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc904c6-c613-4612-bad8-deb4143ad8c0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.411103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc904c6-c613-4612-bad8-deb4143ad8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.436516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d3e8a5fc-4739-40b7-9064-a9bfe4b3a5ac None None] MFA Rules not processed for user `9e88f01e082a4ebbad83de0e02de82b8`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:22.437770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d3e8a5fc-4739-40b7-9064-a9bfe4b3a5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:22.458716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-dcc904c6-c613-4612-bad8-deb4143ad8c0 None None] MFA Rules not processed for user `9f129b7e883d4bf6ad4a9c0a02768aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:22.460900 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dcc904c6-c613-4612-bad8-deb4143ad8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:22.485061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:22.528930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 784/1598] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:22.531447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8a5fc-4739-40b7-9064-a9bfe4b3a5ac None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.545880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8a5fc-4739-40b7-9064-a9bfe4b3a5ac None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.552606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92aa72-a0ec-453f-979f-24a9396be1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.553823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92aa72-a0ec-453f-979f-24a9396be1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.554205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92aa72-a0ec-453f-979f-24a9396be1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.555839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92aa72-a0ec-453f-979f-24a9396be1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.555839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92aa72-a0ec-453f-979f-24a9396be1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.559242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92aa72-a0ec-453f-979f-24a9396be1b8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.561030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 815/1599] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:22.564711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc904c6-c613-4612-bad8-deb4143ad8c0 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.574344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 785/1600] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:22.575067 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92aa72-a0ec-453f-979f-24a9396be1b8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.587314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92aa72-a0ec-453f-979f-24a9396be1b8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.593992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533e4c2-3316-444f-aa00-7843d174b1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.595198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533e4c2-3316-444f-aa00-7843d174b1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.595734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533e4c2-3316-444f-aa00-7843d174b1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.596220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533e4c2-3316-444f-aa00-7843d174b1b1 admin admin] PATH_INFO: `/v3/projects/de12c2563d50474cac799d50041a2fa0/users/f46cbc7de18c4f9fb6f54e96be53fea0/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.628205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533e4c2-3316-444f-aa00-7843d174b1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de12c2563d50474cac799d50041a2fa0, user_id=f46cbc7de18c4f9fb6f54e96be53fea0, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.633190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533e4c2-3316-444f-aa00-7843d174b1b1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.674822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc904c6-c613-4612-bad8-deb4143ad8c0 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.680678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f62a7-c9ee-443d-b311-2152bdfff810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.681343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f62a7-c9ee-443d-b311-2152bdfff810 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.681619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f62a7-c9ee-443d-b311-2152bdfff810 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.681880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f62a7-c9ee-443d-b311-2152bdfff810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.692647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f62a7-c9ee-443d-b311-2152bdfff810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.694981 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f62a7-c9ee-443d-b311-2152bdfff810 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.706084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 816/1601] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:22.707035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f62a7-c9ee-443d-b311-2152bdfff810 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.719529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f62a7-c9ee-443d-b311-2152bdfff810 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.724724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897313e-ee07-4610-aa10-5b047f399078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.726161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897313e-ee07-4610-aa10-5b047f399078 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.726673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897313e-ee07-4610-aa10-5b047f399078 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.726959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897313e-ee07-4610-aa10-5b047f399078 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.727637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897313e-ee07-4610-aa10-5b047f399078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.729326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897313e-ee07-4610-aa10-5b047f399078 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.745030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 786/1602] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:22 2022] PUT /identity/v3/projects/de12c2563d50474cac799d50041a2fa0/users/f46cbc7de18c4f9fb6f54e96be53fea0/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:22.746140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533e4c2-3316-444f-aa00-7843d174b1b1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.753212 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 817/1603] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:22.755312 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897313e-ee07-4610-aa10-5b047f399078 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.774352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897313e-ee07-4610-aa10-5b047f399078 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.839750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533e4c2-3316-444f-aa00-7843d174b1b1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.845503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e57ddc-37cc-4312-9c8e-1a7b3088ac53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.846386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e57ddc-37cc-4312-9c8e-1a7b3088ac53 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.846834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e57ddc-37cc-4312-9c8e-1a7b3088ac53 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.847204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e57ddc-37cc-4312-9c8e-1a7b3088ac53 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.848208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e57ddc-37cc-4312-9c8e-1a7b3088ac53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.850278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e57ddc-37cc-4312-9c8e-1a7b3088ac53 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.854864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ddbc7-ec11-407c-8dea-0c14207e696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.855642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ddbc7-ec11-407c-8dea-0c14207e696e admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.856064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ddbc7-ec11-407c-8dea-0c14207e696e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.856478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ddbc7-ec11-407c-8dea-0c14207e696e admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:22.857690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ddbc7-ec11-407c-8dea-0c14207e696e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:22.860102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ddbc7-ec11-407c-8dea-0c14207e696e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:22.868757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 787/1604] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:22.868757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e57ddc-37cc-4312-9c8e-1a7b3088ac53 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:22.869174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:22.958252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e57ddc-37cc-4312-9c8e-1a7b3088ac53 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:22.964180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed93741-7d4a-411e-936d-b15bc9888199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:22.965115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93741-7d4a-411e-936d-b15bc9888199 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:22.965496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93741-7d4a-411e-936d-b15bc9888199 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:22.965884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93741-7d4a-411e-936d-b15bc9888199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:23.035785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 818/1605] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:22 2022] POST /identity/v3/projects => generated 387 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:23.038266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ddbc7-ec11-407c-8dea-0c14207e696e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:23.045481 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:23.058442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed93741-7d4a-411e-936d-b15bc9888199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:23.069786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed93741-7d4a-411e-936d-b15bc9888199 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:23.077317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 788/1606] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:23.079238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed93741-7d4a-411e-936d-b15bc9888199 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:23.108833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed93741-7d4a-411e-936d-b15bc9888199 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:23.160289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ddbc7-ec11-407c-8dea-0c14207e696e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:23.171412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fa802-693f-40ce-b02c-ee07d1cd76bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:23.172551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fa802-693f-40ce-b02c-ee07d1cd76bc admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:23.174117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fa802-693f-40ce-b02c-ee07d1cd76bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:23.174757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fa802-693f-40ce-b02c-ee07d1cd76bc admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:23.176012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fa802-693f-40ce-b02c-ee07d1cd76bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:23.185293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fa802-693f-40ce-b02c-ee07d1cd76bc admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:23.270523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e870d-8514-4250-8f85-32453396805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:23.272579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e870d-8514-4250-8f85-32453396805d admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:23.272579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e870d-8514-4250-8f85-32453396805d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:23.274126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e870d-8514-4250-8f85-32453396805d admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:23.276388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e870d-8514-4250-8f85-32453396805d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:23.278437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e870d-8514-4250-8f85-32453396805d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:23.304229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 789/1607] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:23.304229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634af867-fbb7-4a53-8f86-d03d9a0beb8f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:23.304229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634af867-fbb7-4a53-8f86-d03d9a0beb8f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:23.304229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634af867-fbb7-4a53-8f86-d03d9a0beb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:23.342619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 819/1608] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:23 2022] POST /identity/v3/projects => generated 363 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:23.343140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-634af867-fbb7-4a53-8f86-d03d9a0beb8f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:23.344409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fa802-693f-40ce-b02c-ee07d1cd76bc admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:23.345639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-634af867-fbb7-4a53-8f86-d03d9a0beb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:23.471216 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 790/1609] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:23.472682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634af867-fbb7-4a53-8f86-d03d9a0beb8f None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:23.488937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634af867-fbb7-4a53-8f86-d03d9a0beb8f None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:23.492665 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fa802-693f-40ce-b02c-ee07d1cd76bc admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:23.499455 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d55f97-4e37-4f71-b642-320dd7f96547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:23.500469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d55f97-4e37-4f71-b642-320dd7f96547 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:23.500906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d55f97-4e37-4f71-b642-320dd7f96547 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:23.501329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d55f97-4e37-4f71-b642-320dd7f96547 admin admin] PATH_INFO: `/v3/projects/c20e7f3aca4c49ae84390a3f1a8ce410/users/864d7f47381e4e4498ad110dbfcfc154/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:23.504421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c1e2a-0f98-43b2-975c-87f0c42e82df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:23.505367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c1e2a-0f98-43b2-975c-87f0c42e82df admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:23.505834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c1e2a-0f98-43b2-975c-87f0c42e82df admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:23.506891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c1e2a-0f98-43b2-975c-87f0c42e82df admin admin] PATH_INFO: `/v3/projects/e31ad35304bc45c4859c43d27288c4d9/users/8b47664214d64d39a435ccf4be3b579a/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:23.541958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c1e2a-0f98-43b2-975c-87f0c42e82df admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31ad35304bc45c4859c43d27288c4d9, user_id=8b47664214d64d39a435ccf4be3b579a, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:23.549967 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d55f97-4e37-4f71-b642-320dd7f96547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20e7f3aca4c49ae84390a3f1a8ce410, user_id=864d7f47381e4e4498ad110dbfcfc154, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:23.553453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c1e2a-0f98-43b2-975c-87f0c42e82df admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:23.562174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d55f97-4e37-4f71-b642-320dd7f96547 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:23.714769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 820/1610] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:23 2022] PUT /identity/v3/projects/c20e7f3aca4c49ae84390a3f1a8ce410/users/864d7f47381e4e4498ad110dbfcfc154/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:23.722121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d55f97-4e37-4f71-b642-320dd7f96547 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:23.744911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 791/1611] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:23 2022] PUT /identity/v3/projects/e31ad35304bc45c4859c43d27288c4d9/users/8b47664214d64d39a435ccf4be3b579a/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:23.746458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c1e2a-0f98-43b2-975c-87f0c42e82df admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:23.814155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:23.821063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d55f97-4e37-4f71-b642-320dd7f96547 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:23.873426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c1e2a-0f98-43b2-975c-87f0c42e82df admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:23.880430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbc691-29bc-4d91-9878-43b814d31084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:23.881728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbc691-29bc-4d91-9878-43b814d31084 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:23.882114 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbc691-29bc-4d91-9878-43b814d31084 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:23.882620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbc691-29bc-4d91-9878-43b814d31084 admin admin] PATH_INFO: `/v3/projects/778299a985204da18b35d86ed8c4c77d/users/9eedc633493b478a9009f7bce07696a8/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:23.921368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:23.932076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbc691-29bc-4d91-9878-43b814d31084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778299a985204da18b35d86ed8c4c77d, user_id=9eedc633493b478a9009f7bce07696a8, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:23.938914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbc691-29bc-4d91-9878-43b814d31084 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:23.950805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c018d1d-51c5-4d04-907f-60cd22c89dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:23.951926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c018d1d-51c5-4d04-907f-60cd22c89dda admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:23.952519 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c018d1d-51c5-4d04-907f-60cd22c89dda admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:23.953013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c018d1d-51c5-4d04-907f-60cd22c89dda admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:23.954284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c018d1d-51c5-4d04-907f-60cd22c89dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:23.958201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c018d1d-51c5-4d04-907f-60cd22c89dda admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:24.033250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 792/1612] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:23 2022] PUT /identity/v3/projects/778299a985204da18b35d86ed8c4c77d/users/9eedc633493b478a9009f7bce07696a8/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:24.034649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbc691-29bc-4d91-9878-43b814d31084 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.076111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 821/1613] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:23 2022] POST /identity/v3/users => generated 294 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:24.078062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c018d1d-51c5-4d04-907f-60cd22c89dda admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.173311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbc691-29bc-4d91-9878-43b814d31084 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:24.182432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd6563-3455-48e7-a72a-10bddc847208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:24.183596 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd6563-3455-48e7-a72a-10bddc847208 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.184142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd6563-3455-48e7-a72a-10bddc847208 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.184598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd6563-3455-48e7-a72a-10bddc847208 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.185560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd6563-3455-48e7-a72a-10bddc847208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:24.188324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd6563-3455-48e7-a72a-10bddc847208 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:24.210541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c018d1d-51c5-4d04-907f-60cd22c89dda admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:24.278337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9ecda-8f25-4828-be26-fa8fc956abd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:24.279328 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9ecda-8f25-4828-be26-fa8fc956abd0 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.285076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9ecda-8f25-4828-be26-fa8fc956abd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.285076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9ecda-8f25-4828-be26-fa8fc956abd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.285315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 793/1614] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:24 2022] POST /identity/v3/users => generated 288 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:24.286583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4943b-18a0-4cb7-8e10-080be962faa9 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.287106 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4943b-18a0-4cb7-8e10-080be962faa9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.287585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4943b-18a0-4cb7-8e10-080be962faa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.298946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9ecda-8f25-4828-be26-fa8fc956abd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:24.301594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9ecda-8f25-4828-be26-fa8fc956abd0 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:24.311028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c7d4943b-18a0-4cb7-8e10-080be962faa9 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:24.312663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c7d4943b-18a0-4cb7-8e10-080be962faa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:24.324264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 794/1615] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:24.324661 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 822/1616] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:24.326079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2797d-95f3-4286-bed8-ac45ebf3e365 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.326270 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df230b06-335a-4ea2-b34a-06811c73e51a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.326582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2797d-95f3-4286-bed8-ac45ebf3e365 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.326771 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df230b06-335a-4ea2-b34a-06811c73e51a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.327009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2797d-95f3-4286-bed8-ac45ebf3e365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.327233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df230b06-335a-4ea2-b34a-06811c73e51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.380387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-12c2797d-95f3-4286-bed8-ac45ebf3e365 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:24.384169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-12c2797d-95f3-4286-bed8-ac45ebf3e365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:24.387386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-df230b06-335a-4ea2-b34a-06811c73e51a None None] MFA Rules not processed for user `8b47664214d64d39a435ccf4be3b579a`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:24.390082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-df230b06-335a-4ea2-b34a-06811c73e51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:24.406597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 823/1617] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:24.406597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2797d-95f3-4286-bed8-ac45ebf3e365 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.427234 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2797d-95f3-4286-bed8-ac45ebf3e365 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:24.434024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f55a0-f9fa-4aee-96b8-0b699ef91cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:24.434721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f55a0-f9fa-4aee-96b8-0b699ef91cca admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.435213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f55a0-f9fa-4aee-96b8-0b699ef91cca admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.435382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f55a0-f9fa-4aee-96b8-0b699ef91cca admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.436492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f55a0-f9fa-4aee-96b8-0b699ef91cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:24.438520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f55a0-f9fa-4aee-96b8-0b699ef91cca admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:24.478104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 824/1618] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:24.480781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f55a0-f9fa-4aee-96b8-0b699ef91cca admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.513394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f55a0-f9fa-4aee-96b8-0b699ef91cca admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:24.533697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0ae76-f9dd-465e-88dd-c9dfac79c4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:24.533948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0ae76-f9dd-465e-88dd-c9dfac79c4fb admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.538077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0ae76-f9dd-465e-88dd-c9dfac79c4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.539122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0ae76-f9dd-465e-88dd-c9dfac79c4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.541533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0ae76-f9dd-465e-88dd-c9dfac79c4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:24.545514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0ae76-f9dd-465e-88dd-c9dfac79c4fb admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:24.584897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 825/1619] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:24.587078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0ae76-f9dd-465e-88dd-c9dfac79c4fb admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.613169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0ae76-f9dd-465e-88dd-c9dfac79c4fb admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:24.622741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 795/1620] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:24.622741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df230b06-335a-4ea2-b34a-06811c73e51a None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.634270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bde41-760e-44a9-9ab8-761ee08b7fee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:24.636321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bde41-760e-44a9-9ab8-761ee08b7fee service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.636742 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bde41-760e-44a9-9ab8-761ee08b7fee service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.637558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bde41-760e-44a9-9ab8-761ee08b7fee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.649313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df230b06-335a-4ea2-b34a-06811c73e51a None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:24.656857 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de3d8d-156e-4b21-8ff4-c75dc455b906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:24.658032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de3d8d-156e-4b21-8ff4-c75dc455b906 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.658565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de3d8d-156e-4b21-8ff4-c75dc455b906 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.659049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de3d8d-156e-4b21-8ff4-c75dc455b906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.659706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bde41-760e-44a9-9ab8-761ee08b7fee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:24.663252 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bde41-760e-44a9-9ab8-761ee08b7fee service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:24.677775 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de3d8d-156e-4b21-8ff4-c75dc455b906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:24.681677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de3d8d-156e-4b21-8ff4-c75dc455b906 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:24.686800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 826/1621] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:24.688532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bde41-760e-44a9-9ab8-761ee08b7fee service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.698603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 796/1622] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:24.700571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de3d8d-156e-4b21-8ff4-c75dc455b906 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.707239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bde41-760e-44a9-9ab8-761ee08b7fee service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:24.714985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de3d8d-156e-4b21-8ff4-c75dc455b906 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:24.719782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1b590-6b4d-4baa-9d42-8da958a6ba3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:24.721692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1b590-6b4d-4baa-9d42-8da958a6ba3e admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.722548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1b590-6b4d-4baa-9d42-8da958a6ba3e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.723370 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1b590-6b4d-4baa-9d42-8da958a6ba3e admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.725916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1b590-6b4d-4baa-9d42-8da958a6ba3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:24.729286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13496e81-fc9f-45a8-b439-4b4eaff1fcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:24.730435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1b590-6b4d-4baa-9d42-8da958a6ba3e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:24.730836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13496e81-fc9f-45a8-b439-4b4eaff1fcd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.731198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13496e81-fc9f-45a8-b439-4b4eaff1fcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.731485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13496e81-fc9f-45a8-b439-4b4eaff1fcd0 admin admin] PATH_INFO: `/v3/projects/cd2593912ce94bf081b26a8d19a820e0/users/e91f95344878496c8afee3aeb108120a/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.736150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:24.755416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13496e81-fc9f-45a8-b439-4b4eaff1fcd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2593912ce94bf081b26a8d19a820e0, user_id=e91f95344878496c8afee3aeb108120a, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:24.759517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13496e81-fc9f-45a8-b439-4b4eaff1fcd0 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:24.761602 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 827/1623] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:24.763591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1b590-6b4d-4baa-9d42-8da958a6ba3e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.781636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1b590-6b4d-4baa-9d42-8da958a6ba3e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:24.786954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565e44d-6454-4ec8-9459-19358c84f0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:24.787773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565e44d-6454-4ec8-9459-19358c84f0ce admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:24.788752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565e44d-6454-4ec8-9459-19358c84f0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:24.788752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565e44d-6454-4ec8-9459-19358c84f0ce admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:24.789075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565e44d-6454-4ec8-9459-19358c84f0ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:24.791796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565e44d-6454-4ec8-9459-19358c84f0ce admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:24.899663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 797/1624] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:24 2022] PUT /identity/v3/projects/cd2593912ce94bf081b26a8d19a820e0/users/e91f95344878496c8afee3aeb108120a/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:24.899663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13496e81-fc9f-45a8-b439-4b4eaff1fcd0 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.906768 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 828/1625] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:24 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:24.910512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565e44d-6454-4ec8-9459-19358c84f0ce admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:24.985747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:25.138260 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13496e81-fc9f-45a8-b439-4b4eaff1fcd0 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:25.153743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e680fe-2939-4d11-a393-39188fa7a8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:25.154119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565e44d-6454-4ec8-9459-19358c84f0ce admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:25.155107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e680fe-2939-4d11-a393-39188fa7a8ad admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:25.155979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e680fe-2939-4d11-a393-39188fa7a8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:25.156575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e680fe-2939-4d11-a393-39188fa7a8ad admin admin] PATH_INFO: `/v3/projects/fbcbc15286a249c1b30b197521f61fb2/users/88769c24fc094b29a29d4a04cfa8c7ab/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:25.163573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7174c5e-9630-4ae9-a1da-0f13bbbaa64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:25.164779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7174c5e-9630-4ae9-a1da-0f13bbbaa64b admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:25.172741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7174c5e-9630-4ae9-a1da-0f13bbbaa64b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:25.173247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7174c5e-9630-4ae9-a1da-0f13bbbaa64b admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:25.175977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7174c5e-9630-4ae9-a1da-0f13bbbaa64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:25.179941 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7174c5e-9630-4ae9-a1da-0f13bbbaa64b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:25.194732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e680fe-2939-4d11-a393-39188fa7a8ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbcbc15286a249c1b30b197521f61fb2, user_id=88769c24fc094b29a29d4a04cfa8c7ab, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:25.202096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e680fe-2939-4d11-a393-39188fa7a8ad admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:25.213019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 829/1626] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:25.213019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7174c5e-9630-4ae9-a1da-0f13bbbaa64b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:25.234430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7174c5e-9630-4ae9-a1da-0f13bbbaa64b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:25.242145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96615c31-646d-45f6-9ca8-91f41411b207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:25.243579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96615c31-646d-45f6-9ca8-91f41411b207 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:25.243820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96615c31-646d-45f6-9ca8-91f41411b207 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:25.244147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96615c31-646d-45f6-9ca8-91f41411b207 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:25.245719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96615c31-646d-45f6-9ca8-91f41411b207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:25.250380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96615c31-646d-45f6-9ca8-91f41411b207 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:25.368819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 798/1627] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:24 2022] PUT /identity/v3/projects/fbcbc15286a249c1b30b197521f61fb2/users/88769c24fc094b29a29d4a04cfa8c7ab/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:25.383070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e680fe-2939-4d11-a393-39188fa7a8ad admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:25.428604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 830/1628] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:25 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:25.428604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ad10-0b5b-4f1d-8dd5-ff0eda58bb24 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:25.428604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ad10-0b5b-4f1d-8dd5-ff0eda58bb24 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:25.428604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ad10-0b5b-4f1d-8dd5-ff0eda58bb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:25.527437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0166ad10-0b5b-4f1d-8dd5-ff0eda58bb24 None None] MFA Rules not processed for user `88769c24fc094b29a29d4a04cfa8c7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:25.535583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0166ad10-0b5b-4f1d-8dd5-ff0eda58bb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:25.564273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e680fe-2939-4d11-a393-39188fa7a8ad admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:25.579617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75b82a-2ef0-42c0-8c1e-8470258b31d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:25.581100 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75b82a-2ef0-42c0-8c1e-8470258b31d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:25.581569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75b82a-2ef0-42c0-8c1e-8470258b31d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:25.582030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75b82a-2ef0-42c0-8c1e-8470258b31d2 admin admin] PATH_INFO: `/v3/projects/cd2593912ce94bf081b26a8d19a820e0/users/e91f95344878496c8afee3aeb108120a/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:25.589233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75b82a-2ef0-42c0-8c1e-8470258b31d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2593912ce94bf081b26a8d19a820e0, user_id=e91f95344878496c8afee3aeb108120a, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:25.595305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75b82a-2ef0-42c0-8c1e-8470258b31d2 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:25.643789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 799/1629] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:25 2022] PUT /identity/v3/projects/cd2593912ce94bf081b26a8d19a820e0/users/e91f95344878496c8afee3aeb108120a/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:25.649370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75b82a-2ef0-42c0-8c1e-8470258b31d2 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:25.731682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 831/1630] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:25.731682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ed0fe-1e15-45a0-8cce-1b163fc840c6 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:25.731682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ed0fe-1e15-45a0-8cce-1b163fc840c6 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:25.731682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ed0fe-1e15-45a0-8cce-1b163fc840c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:25.790018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6d8ed0fe-1e15-45a0-8cce-1b163fc840c6 None None] MFA Rules not processed for user `e91f95344878496c8afee3aeb108120a`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:25.791745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6d8ed0fe-1e15-45a0-8cce-1b163fc840c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:25.835320 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75b82a-2ef0-42c0-8c1e-8470258b31d2 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:25.841027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f14b2-8ad8-4012-b0f0-805706b0f3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:25.841676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:25.842337 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f14b2-8ad8-4012-b0f0-805706b0f3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:25.842645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f14b2-8ad8-4012-b0f0-805706b0f3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:25.842914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f14b2-8ad8-4012-b0f0-805706b0f3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:25.843589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f14b2-8ad8-4012-b0f0-805706b0f3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:25.845339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f14b2-8ad8-4012-b0f0-805706b0f3f9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:25.859695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 800/1631] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:25.860610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f14b2-8ad8-4012-b0f0-805706b0f3f9 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:25.871701 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f14b2-8ad8-4012-b0f0-805706b0f3f9 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:25.877967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406f371-8522-482d-85d4-d533985d5955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:25.878634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406f371-8522-482d-85d4-d533985d5955 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:25.878911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406f371-8522-482d-85d4-d533985d5955 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:25.879185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406f371-8522-482d-85d4-d533985d5955 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:25.879825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406f371-8522-482d-85d4-d533985d5955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:25.882041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406f371-8522-482d-85d4-d533985d5955 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:25.909646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 832/1632] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:25.914837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ed0fe-1e15-45a0-8cce-1b163fc840c6 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:25.923550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ed0fe-1e15-45a0-8cce-1b163fc840c6 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:25.929237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d4883-b857-49b8-b2f6-ed3fa4863af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:25.930020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d4883-b857-49b8-b2f6-ed3fa4863af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:25.930376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d4883-b857-49b8-b2f6-ed3fa4863af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:25.930775 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d4883-b857-49b8-b2f6-ed3fa4863af8 admin admin] PATH_INFO: `/v3/projects/a7369958d177413c9ca384526dfd5278/users/d7917a215e4b43929fc3aa3aa303ffe3/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:25.961987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d4883-b857-49b8-b2f6-ed3fa4863af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7369958d177413c9ca384526dfd5278, user_id=d7917a215e4b43929fc3aa3aa303ffe3, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:25.965832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d4883-b857-49b8-b2f6-ed3fa4863af8 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:26.036914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 801/1633] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:25 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:26.039219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406f371-8522-482d-85d4-d533985d5955 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:26.088460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:26.500793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406f371-8522-482d-85d4-d533985d5955 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:26.500793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb3ac9-1c08-4923-bc21-4f14108c56fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:26.500793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3ac9-1c08-4923-bc21-4f14108c56fb service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:26.500793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3ac9-1c08-4923-bc21-4f14108c56fb service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:26.504093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3ac9-1c08-4923-bc21-4f14108c56fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:26.504254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 833/1634] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:25 2022] PUT /identity/v3/projects/a7369958d177413c9ca384526dfd5278/users/d7917a215e4b43929fc3aa3aa303ffe3/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:26.504254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d4883-b857-49b8-b2f6-ed3fa4863af8 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:26.504730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb3ac9-1c08-4923-bc21-4f14108c56fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:26.504730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb3ac9-1c08-4923-bc21-4f14108c56fb service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:26.504730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 802/1635] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:26.504730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb3ac9-1c08-4923-bc21-4f14108c56fb service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:26.505044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d4883-b857-49b8-b2f6-ed3fa4863af8 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:26.505044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107205f-e959-4ab0-80aa-1f4b72b7cbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:26.505044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107205f-e959-4ab0-80aa-1f4b72b7cbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:26.505044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107205f-e959-4ab0-80aa-1f4b72b7cbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:26.505384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107205f-e959-4ab0-80aa-1f4b72b7cbc2 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:26.505384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107205f-e959-4ab0-80aa-1f4b72b7cbc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:26.511522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb3ac9-1c08-4923-bc21-4f14108c56fb service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:26.511699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107205f-e959-4ab0-80aa-1f4b72b7cbc2 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:26.512387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a62f4-a6eb-43c9-bf58-8359fe4a9a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:26.512387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a62f4-a6eb-43c9-bf58-8359fe4a9a1b admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:26.512387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a62f4-a6eb-43c9-bf58-8359fe4a9a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:26.512387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a62f4-a6eb-43c9-bf58-8359fe4a9a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:26.512838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a62f4-a6eb-43c9-bf58-8359fe4a9a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:26.512838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a62f4-a6eb-43c9-bf58-8359fe4a9a1b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:26.512838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 803/1636] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:26.512838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a62f4-a6eb-43c9-bf58-8359fe4a9a1b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:26.512838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a62f4-a6eb-43c9-bf58-8359fe4a9a1b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:26.524499 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e57ba-5c26-4968-abde-b7a38e920b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:26.525734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e57ba-5c26-4968-abde-b7a38e920b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:26.526242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e57ba-5c26-4968-abde-b7a38e920b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:26.529363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e57ba-5c26-4968-abde-b7a38e920b7f admin admin] PATH_INFO: `/v3/projects/a7369958d177413c9ca384526dfd5278/users/d7917a215e4b43929fc3aa3aa303ffe3/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:26.547820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e57ba-5c26-4968-abde-b7a38e920b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7369958d177413c9ca384526dfd5278, user_id=d7917a215e4b43929fc3aa3aa303ffe3, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:26.547820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e57ba-5c26-4968-abde-b7a38e920b7f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:26.567180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 804/1637] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:26 2022] PUT /identity/v3/projects/a7369958d177413c9ca384526dfd5278/users/d7917a215e4b43929fc3aa3aa303ffe3/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:26:26.588410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0984b4-0621-49ea-bae9-1e6f0dac5c86 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:26.589228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0984b4-0621-49ea-bae9-1e6f0dac5c86 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:26.589815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0984b4-0621-49ea-bae9-1e6f0dac5c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:26.671978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-9c0984b4-0621-49ea-bae9-1e6f0dac5c86 None None] MFA Rules not processed for user `d7917a215e4b43929fc3aa3aa303ffe3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:26.675143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9c0984b4-0621-49ea-bae9-1e6f0dac5c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:26.683205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 834/1638] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:26 2022] POST /identity/v3/users => generated 300 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:26.684998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107205f-e959-4ab0-80aa-1f4b72b7cbc2 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:26.709701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107205f-e959-4ab0-80aa-1f4b72b7cbc2 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:26.876673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 805/1639] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:26:26.911717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee08546-7a1e-458c-afbc-fc431d63c0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:26.920613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0984b4-0621-49ea-bae9-1e6f0dac5c86 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:26.926090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee08546-7a1e-458c-afbc-fc431d63c0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:26.926469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee08546-7a1e-458c-afbc-fc431d63c0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:26.926950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee08546-7a1e-458c-afbc-fc431d63c0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:26.931536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee08546-7a1e-458c-afbc-fc431d63c0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:26.931536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee08546-7a1e-458c-afbc-fc431d63c0a8 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:26.991093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 835/1640] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:26.991093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee08546-7a1e-458c-afbc-fc431d63c0a8 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:27.008349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee08546-7a1e-458c-afbc-fc431d63c0a8 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:27.019312 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa3197-146c-40c9-ae5d-42b6e3821f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:27.020046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa3197-146c-40c9-ae5d-42b6e3821f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:27.020330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa3197-146c-40c9-ae5d-42b6e3821f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:27.020605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa3197-146c-40c9-ae5d-42b6e3821f3d admin admin] PATH_INFO: `/v3/projects/9df2e90531504e20bd9c1461f8d3f126/users/def8351bd62947ef9dfd10374f54dca4/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:27.055717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa3197-146c-40c9-ae5d-42b6e3821f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df2e90531504e20bd9c1461f8d3f126, user_id=def8351bd62947ef9dfd10374f54dca4, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:27.063893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa3197-146c-40c9-ae5d-42b6e3821f3d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:27.073006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0984b4-0621-49ea-bae9-1e6f0dac5c86 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:27.078018 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110966-ebfb-435d-92c9-ab5dbe4767cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:27.078782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110966-ebfb-435d-92c9-ab5dbe4767cb service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:27.079126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110966-ebfb-435d-92c9-ab5dbe4767cb service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:27.079466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110966-ebfb-435d-92c9-ab5dbe4767cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:27.093245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da110966-ebfb-435d-92c9-ab5dbe4767cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:27.099095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da110966-ebfb-435d-92c9-ab5dbe4767cb service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:27.113763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 806/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:27.250412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 836/1642] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:26 2022] PUT /identity/v3/projects/9df2e90531504e20bd9c1461f8d3f126/users/def8351bd62947ef9dfd10374f54dca4/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:27.254190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110966-ebfb-435d-92c9-ab5dbe4767cb service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:27.277230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110966-ebfb-435d-92c9-ab5dbe4767cb service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:27.394784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8ead6-ef53-439e-b3bd-7e08c6cded4e None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:27.394784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8ead6-ef53-439e-b3bd-7e08c6cded4e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:27.395431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8ead6-ef53-439e-b3bd-7e08c6cded4e None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:27.397002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 837/1643] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 22 17:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:26:27.404386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2baf6-0561-46e9-a09a-559738118b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:27.405634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2baf6-0561-46e9-a09a-559738118b02 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:27.406110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2baf6-0561-46e9-a09a-559738118b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:27.407151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2baf6-0561-46e9-a09a-559738118b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:27.408419 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2baf6-0561-46e9-a09a-559738118b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:27.415732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2baf6-0561-46e9-a09a-559738118b02 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:27.425218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161aaf0f-371e-4d3d-bea4-6fc520a00493 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:27.425573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161aaf0f-371e-4d3d-bea4-6fc520a00493 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:27.425944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161aaf0f-371e-4d3d-bea4-6fc520a00493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:27.432345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 807/1644] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:27.437517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2baf6-0561-46e9-a09a-559738118b02 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:27.448357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:27.461240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2baf6-0561-46e9-a09a-559738118b02 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:27.474140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79681b-1881-4755-aec0-2289dab07f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:27.480732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79681b-1881-4755-aec0-2289dab07f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:27.480732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79681b-1881-4755-aec0-2289dab07f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:27.480732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79681b-1881-4755-aec0-2289dab07f38 admin admin] PATH_INFO: `/v3/projects/9df2e90531504e20bd9c1461f8d3f126/users/def8351bd62947ef9dfd10374f54dca4/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:27.483837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79681b-1881-4755-aec0-2289dab07f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df2e90531504e20bd9c1461f8d3f126, user_id=def8351bd62947ef9dfd10374f54dca4, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:27.488906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79681b-1881-4755-aec0-2289dab07f38 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:27.526646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 808/1645] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:27 2022] PUT /identity/v3/projects/9df2e90531504e20bd9c1461f8d3f126/users/def8351bd62947ef9dfd10374f54dca4/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:27.527222 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-161aaf0f-371e-4d3d-bea4-6fc520a00493 None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:27.528309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-161aaf0f-371e-4d3d-bea4-6fc520a00493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:27.543574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1076764-2442-42d5-8306-a5d47b934767 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:27.546883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1076764-2442-42d5-8306-a5d47b934767 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:27.548545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1076764-2442-42d5-8306-a5d47b934767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:27.636814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-f1076764-2442-42d5-8306-a5d47b934767 None None] MFA Rules not processed for user `def8351bd62947ef9dfd10374f54dca4`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:27.638188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f1076764-2442-42d5-8306-a5d47b934767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:27.718309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 838/1646] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 22 17:26:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:26:27.735757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0e443-a099-4994-acef-0ace39d9b0b4 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:27.735757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0e443-a099-4994-acef-0ace39d9b0b4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:27.738523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0e443-a099-4994-acef-0ace39d9b0b4 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:27.750175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 839/1647] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 22 17:26:27 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:26:27.757544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cee731-2ed4-4617-b5d7-497d4de008de None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:27.758378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cee731-2ed4-4617-b5d7-497d4de008de None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:27.760440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cee731-2ed4-4617-b5d7-497d4de008de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:27.829969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 809/1648] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:27.840740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-14cee731-2ed4-4617-b5d7-497d4de008de None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:27.846473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-14cee731-2ed4-4617-b5d7-497d4de008de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:27.860006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:27.876032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 840/1649] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:26:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:27.895811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1076764-2442-42d5-8306-a5d47b934767 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:27.972333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cee731-2ed4-4617-b5d7-497d4de008de None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:28.043214 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1076764-2442-42d5-8306-a5d47b934767 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:28.054143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b1c9d-16e3-4c87-9a09-fdd5a6fa81e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:28.055741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b1c9d-16e3-4c87-9a09-fdd5a6fa81e4 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:28.055741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b1c9d-16e3-4c87-9a09-fdd5a6fa81e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:28.055741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b1c9d-16e3-4c87-9a09-fdd5a6fa81e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:28.087699 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b1c9d-16e3-4c87-9a09-fdd5a6fa81e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:28.091274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b1c9d-16e3-4c87-9a09-fdd5a6fa81e4 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:28.127720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 810/1650] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:28.138177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cee731-2ed4-4617-b5d7-497d4de008de None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:28.145848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5520025-6d83-4801-8e42-2b60c81abba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:28.148292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5520025-6d83-4801-8e42-2b60c81abba0 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:28.149192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5520025-6d83-4801-8e42-2b60c81abba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:28.150091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5520025-6d83-4801-8e42-2b60c81abba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:28.181077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5520025-6d83-4801-8e42-2b60c81abba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:28.184583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5520025-6d83-4801-8e42-2b60c81abba0 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:28.231290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 841/1651] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:29.427154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b1c9d-16e3-4c87-9a09-fdd5a6fa81e4 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:29.611847 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b1c9d-16e3-4c87-9a09-fdd5a6fa81e4 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:29.628438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:29.628688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:29.629108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:29.629527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] PATH_INFO: `/v3/users/9f129b7e883d4bf6ad4a9c0a02768aa0` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:29.636125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f129b7e883d4bf6ad4a9c0a02768aa0)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:29.638715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:29.665620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:29.987729 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f129b7e883d4bf6ad4a9c0a02768aa0'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:29.993918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f129b7e883d4bf6ad4a9c0a02768aa0'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:29.995533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f129b7e883d4bf6ad4a9c0a02768aa0'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:30.017577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f129b7e883d4bf6ad4a9c0a02768aa0'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:30.069621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f129b7e883d4bf6ad4a9c0a02768aa0'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:30.204975 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 811/1652] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:29 2022] DELETE /identity/v3/users/9f129b7e883d4bf6ad4a9c0a02768aa0 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:30.212882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55047204-d3a5-46bf-b87c-eb9fe8c23a39 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:30.213788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55047204-d3a5-46bf-b87c-eb9fe8c23a39 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:30.213962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55047204-d3a5-46bf-b87c-eb9fe8c23a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:30.280374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-55047204-d3a5-46bf-b87c-eb9fe8c23a39 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:30.282772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-55047204-d3a5-46bf-b87c-eb9fe8c23a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:30.474922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:30.477486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:30.524754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 842/1653] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:26:30.567763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55047204-d3a5-46bf-b87c-eb9fe8c23a39 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:30.570311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-55047204-d3a5-46bf-b87c-eb9fe8c23a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:30.631820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:30.656997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28aae2-c5ff-4553-8359-c527c8170e21 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:30.669887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:30.669887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:30.670207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:30.670543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] PATH_INFO: `/v3/users/8b47664214d64d39a435ccf4be3b579a` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:30.671943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b47664214d64d39a435ccf4be3b579a)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:30.674208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:30.711936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55047204-d3a5-46bf-b87c-eb9fe8c23a39 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:30.716370 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff7e0-9480-4f47-87e7-4f54c8124cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:30.717011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff7e0-9480-4f47-87e7-4f54c8124cdf service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:30.717287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff7e0-9480-4f47-87e7-4f54c8124cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:30.717559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff7e0-9480-4f47-87e7-4f54c8124cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:30.744102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cff7e0-9480-4f47-87e7-4f54c8124cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:30.745642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cff7e0-9480-4f47-87e7-4f54c8124cdf service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:30.769278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 843/1654] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:26:30.991077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b47664214d64d39a435ccf4be3b579a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:30.992279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b47664214d64d39a435ccf4be3b579a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:30.993369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b47664214d64d39a435ccf4be3b579a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:31.010168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b47664214d64d39a435ccf4be3b579a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:31.037759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e9042711-bffa-4f2a-aaf0-9b3f70c2a1c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b47664214d64d39a435ccf4be3b579a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:31.136641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff7e0-9480-4f47-87e7-4f54c8124cdf service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:31.138353 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-01cff7e0-9480-4f47-87e7-4f54c8124cdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:31.171259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 812/1655] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:30 2022] DELETE /identity/v3/users/8b47664214d64d39a435ccf4be3b579a => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:31.177673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42955c97-05d8-4250-9430-fc3cf8295e82 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:31.178975 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42955c97-05d8-4250-9430-fc3cf8295e82 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:31.179469 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42955c97-05d8-4250-9430-fc3cf8295e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:31.232322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-42955c97-05d8-4250-9430-fc3cf8295e82 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:31.233843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-42955c97-05d8-4250-9430-fc3cf8295e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:31.309346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff7e0-9480-4f47-87e7-4f54c8124cdf service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:31.391291 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 813/1656] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:31.393044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcac2d-d70c-4b16-9d2a-65252f7a8181 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:31.393569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcac2d-d70c-4b16-9d2a-65252f7a8181 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:31.394042 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcac2d-d70c-4b16-9d2a-65252f7a8181 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:31.395878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 814/1657] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 22 17:26:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:26:31.403406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df537b1-f0c2-4159-8817-5dcf89b7f6ad None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:31.410925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df537b1-f0c2-4159-8817-5dcf89b7f6ad None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:31.410925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df537b1-f0c2-4159-8817-5dcf89b7f6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:31.445237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1df537b1-f0c2-4159-8817-5dcf89b7f6ad None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:31.446980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1df537b1-f0c2-4159-8817-5dcf89b7f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:31.468535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0946a13-e79f-41c6-a9c6-bbec97a490d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:31.469621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0946a13-e79f-41c6-a9c6-bbec97a490d8 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:31.470118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0946a13-e79f-41c6-a9c6-bbec97a490d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:31.470633 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0946a13-e79f-41c6-a9c6-bbec97a490d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:31.474392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d0946a13-e79f-41c6-a9c6-bbec97a490d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:31.580882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 815/1658] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 22 17:26:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:26:31.583208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df537b1-f0c2-4159-8817-5dcf89b7f6ad None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:31.588249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1df537b1-f0c2-4159-8817-5dcf89b7f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:31.653565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0946a13-e79f-41c6-a9c6-bbec97a490d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:31.655562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0946a13-e79f-41c6-a9c6-bbec97a490d8 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:31.674853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 844/1659] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:26:31.676280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246aa67f-b0c9-4319-924d-e053c0cd3a9b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:31.676541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246aa67f-b0c9-4319-924d-e053c0cd3a9b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:31.676843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246aa67f-b0c9-4319-924d-e053c0cd3a9b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:31.683960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 845/1660] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 22 17:26:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:26:31.685312 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff884ff-6700-4d84-bdb5-a0f84cb88f65 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:31.685627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff884ff-6700-4d84-bdb5-a0f84cb88f65 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:31.685932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff884ff-6700-4d84-bdb5-a0f84cb88f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:31.712637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-aff884ff-6700-4d84-bdb5-a0f84cb88f65 None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:31.714062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-aff884ff-6700-4d84-bdb5-a0f84cb88f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:31.723820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:31.732952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 846/1661] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:26:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:31.751766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff884ff-6700-4d84-bdb5-a0f84cb88f65 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:31.753969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-aff884ff-6700-4d84-bdb5-a0f84cb88f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:31.780289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff884ff-6700-4d84-bdb5-a0f84cb88f65 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:31.794129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075031-f146-45ec-833f-bb594b50dcbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:31.794886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075031-f146-45ec-833f-bb594b50dcbe service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:31.796513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075031-f146-45ec-833f-bb594b50dcbe service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:31.797471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075031-f146-45ec-833f-bb594b50dcbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:31.820928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df537b1-f0c2-4159-8817-5dcf89b7f6ad None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:31.823251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075031-f146-45ec-833f-bb594b50dcbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:31.823425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075031-f146-45ec-833f-bb594b50dcbe service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:31.837334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544cf44-4836-41cb-8ae7-d412bdf9463f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:31.837334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544cf44-4836-41cb-8ae7-d412bdf9463f service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:31.837738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544cf44-4836-41cb-8ae7-d412bdf9463f service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:31.838099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544cf44-4836-41cb-8ae7-d412bdf9463f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:31.851590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 847/1662] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:26:31.898029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544cf44-4836-41cb-8ae7-d412bdf9463f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:31.900985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544cf44-4836-41cb-8ae7-d412bdf9463f service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:31.939261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 816/1663] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:33.762411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075031-f146-45ec-833f-bb594b50dcbe service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:33.764107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-75075031-f146-45ec-833f-bb594b50dcbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:33.786773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075031-f146-45ec-833f-bb594b50dcbe service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:33.805058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:33.805058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:33.805058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:33.805058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] PATH_INFO: `/v3/projects/198c51c799bf40e98617b64389fc175c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:33.805383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] RBAC: Authorizing `identity:delete_project(project_id=198c51c799bf40e98617b64389fc175c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:33.805383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:33.840461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:34.086792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '198c51c799bf40e98617b64389fc175c', 'request_id': 'req-446a42f7-e90a-4e94-87cc-0a0c6134016c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:34.100919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '198c51c799bf40e98617b64389fc175c', 'request_id': 'req-446a42f7-e90a-4e94-87cc-0a0c6134016c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:34.121701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '198c51c799bf40e98617b64389fc175c', 'request_id': 'req-446a42f7-e90a-4e94-87cc-0a0c6134016c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:34.288523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] The token cache is being invalidate because project 198c51c799bf40e98617b64389fc175c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:26:34.289127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:34.290716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:34.297230 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 848/1664] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:26:33 2022] DELETE /identity/v3/projects/198c51c799bf40e98617b64389fc175c => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:34.302149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544cf44-4836-41cb-8ae7-d412bdf9463f service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:34.324021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544cf44-4836-41cb-8ae7-d412bdf9463f service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:34.459919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:34.575119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:34.578856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:34.579487 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:34.582632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] PATH_INFO: `/v3/users/e91f95344878496c8afee3aeb108120a` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:34.592188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e91f95344878496c8afee3aeb108120a)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:34.605423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:34.658993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:34.660957 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:34.708472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a42f7-e90a-4e94-87cc-0a0c6134016c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:34.746370 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:34.759780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:34.763483 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:34.765594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] PATH_INFO: `/v3/projects/e31ad35304bc45c4859c43d27288c4d9` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:34.768684 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e31ad35304bc45c4859c43d27288c4d9)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:34.776927 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:34.969044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e91f95344878496c8afee3aeb108120a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:34.970617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e31ad35304bc45c4859c43d27288c4d9', 'request_id': 'req-d8839b84-0c8e-469f-b1c8-53e46104d64d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:34.971948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e91f95344878496c8afee3aeb108120a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:34.978068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e91f95344878496c8afee3aeb108120a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:34.981553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e31ad35304bc45c4859c43d27288c4d9', 'request_id': 'req-d8839b84-0c8e-469f-b1c8-53e46104d64d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:34.990622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e31ad35304bc45c4859c43d27288c4d9', 'request_id': 'req-d8839b84-0c8e-469f-b1c8-53e46104d64d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:35.005227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e91f95344878496c8afee3aeb108120a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:35.026154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e91f95344878496c8afee3aeb108120a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:35.102093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] The token cache is being invalidate because project e31ad35304bc45c4859c43d27288c4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:26:35.103880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:35.103880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d8839b84-0c8e-469f-b1c8-53e46104d64d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:35.106116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 849/1665] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:26:34 2022] DELETE /identity/v3/projects/e31ad35304bc45c4859c43d27288c4d9 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:35.110273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43110ce1-5abc-4f37-aead-8c018e6911af None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:35.110977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43110ce1-5abc-4f37-aead-8c018e6911af None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:35.111273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43110ce1-5abc-4f37-aead-8c018e6911af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:35.165520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-43110ce1-5abc-4f37-aead-8c018e6911af None None] MFA Rules not processed for user `b0db362075b04b2294a6268576a5bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:35.167385 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-43110ce1-5abc-4f37-aead-8c018e6911af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:35.225533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 817/1666] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:34 2022] DELETE /identity/v3/users/e91f95344878496c8afee3aeb108120a => generated 0 bytes in 916 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:35.232401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:35.247709 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:35.410512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd970c-1224-406c-b64c-6613fc7b1da4 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:35.426906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:35.428407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:35.428874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:35.429330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] PATH_INFO: `/v3/users/d7917a215e4b43929fc3aa3aa303ffe3` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:35.434814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7917a215e4b43929fc3aa3aa303ffe3)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:35.438071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:35.475552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 850/1667] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:35.485726 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43110ce1-5abc-4f37-aead-8c018e6911af None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:35.488911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-43110ce1-5abc-4f37-aead-8c018e6911af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:35.540286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:35.862842 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43110ce1-5abc-4f37-aead-8c018e6911af None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:35.899987 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7917a215e4b43929fc3aa3aa303ffe3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:35.903356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7917a215e4b43929fc3aa3aa303ffe3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:35.911357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7917a215e4b43929fc3aa3aa303ffe3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:35.936830 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7917a215e4b43929fc3aa3aa303ffe3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:35.994626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7917a215e4b43929fc3aa3aa303ffe3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:36.124732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 818/1668] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:35 2022] DELETE /identity/v3/users/d7917a215e4b43929fc3aa3aa303ffe3 => generated 0 bytes in 899 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:36.226718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f030-01c3-4a0b-9357-27c1664cc4e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:36.227850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f030-01c3-4a0b-9357-27c1664cc4e9 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:36.228352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f030-01c3-4a0b-9357-27c1664cc4e9 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:36.228793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f030-01c3-4a0b-9357-27c1664cc4e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:36.245697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1059f030-01c3-4a0b-9357-27c1664cc4e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:36.524809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059f030-01c3-4a0b-9357-27c1664cc4e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:36.526727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059f030-01c3-4a0b-9357-27c1664cc4e9 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:36.551109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 851/1669] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 1067 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:36.899479 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:36.976845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:36.978715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:37.136678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73129c2b-c85a-4d51-ab66-5d86d207ecfc admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:37.159376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:37.161262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:37.162928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:37.162928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] PATH_INFO: `/v3/projects/cd2593912ce94bf081b26a8d19a820e0` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:37.168392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd2593912ce94bf081b26a8d19a820e0)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:37.173413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: WARNING py.warnings [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:26:37.173413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: warnings.warn(msg) Aug 22 17:26:37.173413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: Aug 22 17:26:37.174996 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:37.456168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd2593912ce94bf081b26a8d19a820e0', 'request_id': 'req-27132dc8-ab11-409a-a325-38af9dbb06b2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:37.530094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd2593912ce94bf081b26a8d19a820e0', 'request_id': 'req-27132dc8-ab11-409a-a325-38af9dbb06b2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:37.659773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd2593912ce94bf081b26a8d19a820e0', 'request_id': 'req-27132dc8-ab11-409a-a325-38af9dbb06b2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:37.691529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] The token cache is being invalidate because project cd2593912ce94bf081b26a8d19a820e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:26:37.692054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:37.693190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-27132dc8-ab11-409a-a325-38af9dbb06b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:37.695584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 819/1670] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:26:36 2022] DELETE /identity/v3/projects/cd2593912ce94bf081b26a8d19a820e0 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:37.701271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a8d46-7daa-457c-a169-f3519743a843 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:37.702289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a8d46-7daa-457c-a169-f3519743a843 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:37.702954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a8d46-7daa-457c-a169-f3519743a843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:37.738657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-621a8d46-7daa-457c-a169-f3519743a843 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:37.741180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-621a8d46-7daa-457c-a169-f3519743a843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:37.908176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 852/1671] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:26:37.934095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1217e26-dc4b-4b40-ad1b-188d54d1fa83 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:37.935359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1217e26-dc4b-4b40-ad1b-188d54d1fa83 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:37.936158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1217e26-dc4b-4b40-ad1b-188d54d1fa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:38.045196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c1217e26-dc4b-4b40-ad1b-188d54d1fa83 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:38.052084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c1217e26-dc4b-4b40-ad1b-188d54d1fa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:38.116270 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 820/1672] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:38.122429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a8d46-7daa-457c-a169-f3519743a843 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:38.148786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a8d46-7daa-457c-a169-f3519743a843 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:38.163814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7b382-53d2-4266-b23e-f4cc0f97a980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:38.165104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7b382-53d2-4266-b23e-f4cc0f97a980 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:38.166767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7b382-53d2-4266-b23e-f4cc0f97a980 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:38.167708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7b382-53d2-4266-b23e-f4cc0f97a980 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:38.169267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7b382-53d2-4266-b23e-f4cc0f97a980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:38.170271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7b382-53d2-4266-b23e-f4cc0f97a980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:38.177238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7b382-53d2-4266-b23e-f4cc0f97a980 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:38.216849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 853/1673] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:38.231071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1217e26-dc4b-4b40-ad1b-188d54d1fa83 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:38.251787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1217e26-dc4b-4b40-ad1b-188d54d1fa83 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:38.275951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cca37-ef81-4ab3-a31d-b3c67e611f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:38.277082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cca37-ef81-4ab3-a31d-b3c67e611f5f admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:38.277509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cca37-ef81-4ab3-a31d-b3c67e611f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:38.277923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cca37-ef81-4ab3-a31d-b3c67e611f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:38.278793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cca37-ef81-4ab3-a31d-b3c67e611f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:38.281120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cca37-ef81-4ab3-a31d-b3c67e611f5f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:38.311236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:38.350962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7b382-53d2-4266-b23e-f4cc0f97a980 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:38.351473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-51e7b382-53d2-4266-b23e-f4cc0f97a980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:38.416880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7b382-53d2-4266-b23e-f4cc0f97a980 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:38.451977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:38.451977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:38.451977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:38.451977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] PATH_INFO: `/v3/projects/a7369958d177413c9ca384526dfd5278` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:38.452853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 821/1674] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:38 2022] POST /identity/v3/projects => generated 377 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:38.453981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cca37-ef81-4ab3-a31d-b3c67e611f5f admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:38.468400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7369958d177413c9ca384526dfd5278)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:38.514041 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:38.760675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7369958d177413c9ca384526dfd5278', 'request_id': 'req-7498ffd0-3221-49e0-8b1e-6f5906799962'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:38.775197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7369958d177413c9ca384526dfd5278', 'request_id': 'req-7498ffd0-3221-49e0-8b1e-6f5906799962'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:38.780864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cca37-ef81-4ab3-a31d-b3c67e611f5f admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:38.792840 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7369958d177413c9ca384526dfd5278', 'request_id': 'req-7498ffd0-3221-49e0-8b1e-6f5906799962'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:38.917842 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] The token cache is being invalidate because project a7369958d177413c9ca384526dfd5278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:26:38.918393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:38.922311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7498ffd0-3221-49e0-8b1e-6f5906799962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:38.928472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 854/1675] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:26:38 2022] DELETE /identity/v3/projects/a7369958d177413c9ca384526dfd5278 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:38.937956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949c858-2679-40bc-aba6-dbc3dc8263e4 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:38.938243 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949c858-2679-40bc-aba6-dbc3dc8263e4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:38.938560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949c858-2679-40bc-aba6-dbc3dc8263e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:38.961742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f18cd-4977-42dc-8149-70f01ad39915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:38.965888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f18cd-4977-42dc-8149-70f01ad39915 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:38.970659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f18cd-4977-42dc-8149-70f01ad39915 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:38.971373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f18cd-4977-42dc-8149-70f01ad39915 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:38.976425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f18cd-4977-42dc-8149-70f01ad39915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:38.989237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8949c858-2679-40bc-aba6-dbc3dc8263e4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:39.001709 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f18cd-4977-42dc-8149-70f01ad39915 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:39.002084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8949c858-2679-40bc-aba6-dbc3dc8263e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:39.030798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:39.047131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 855/1676] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:39.054301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456073c-1951-4cfc-b672-582b6b23b1d6 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:39.054301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456073c-1951-4cfc-b672-582b6b23b1d6 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:39.054584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456073c-1951-4cfc-b672-582b6b23b1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:39.097887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-b456073c-1951-4cfc-b672-582b6b23b1d6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:39.099285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b456073c-1951-4cfc-b672-582b6b23b1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:39.126667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 856/1677] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:26:39.135206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456073c-1951-4cfc-b672-582b6b23b1d6 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:39.163942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456073c-1951-4cfc-b672-582b6b23b1d6 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:39.184421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c3934-a002-4736-a851-2a9bafe52f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:39.185380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c3934-a002-4736-a851-2a9bafe52f74 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:39.186316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c3934-a002-4736-a851-2a9bafe52f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:39.186790 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c3934-a002-4736-a851-2a9bafe52f74 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:39.188248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c3934-a002-4736-a851-2a9bafe52f74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:39.188803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c3934-a002-4736-a851-2a9bafe52f74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:39.202179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c3934-a002-4736-a851-2a9bafe52f74 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:39.208022 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 822/1678] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:38 2022] POST /identity/v3/users => generated 289 bytes in 754 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:39.213988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f18cd-4977-42dc-8149-70f01ad39915 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:39.220401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-6e9f18cd-4977-42dc-8149-70f01ad39915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:39.234603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 857/1679] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:26:39.246607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c3934-a002-4736-a851-2a9bafe52f74 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:39.259858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c3934-a002-4736-a851-2a9bafe52f74 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:39.265365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca6c72-063b-4198-ad0a-b389420020f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:39.266119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca6c72-063b-4198-ad0a-b389420020f4 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:39.266503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca6c72-063b-4198-ad0a-b389420020f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:39.267068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca6c72-063b-4198-ad0a-b389420020f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:39.267821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca6c72-063b-4198-ad0a-b389420020f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:39.269944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca6c72-063b-4198-ad0a-b389420020f4 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:39.283212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f18cd-4977-42dc-8149-70f01ad39915 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:39.300599 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1452ae2-e941-4374-8379-3d8776e9bb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:39.301839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452ae2-e941-4374-8379-3d8776e9bb3b admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:39.306367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452ae2-e941-4374-8379-3d8776e9bb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:39.307897 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452ae2-e941-4374-8379-3d8776e9bb3b admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:39.308125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1452ae2-e941-4374-8379-3d8776e9bb3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:39.310520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1452ae2-e941-4374-8379-3d8776e9bb3b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:39.339620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 823/1680] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:26:39.342793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1452ae2-e941-4374-8379-3d8776e9bb3b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:39.367830 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1452ae2-e941-4374-8379-3d8776e9bb3b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:39.378294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 858/1681] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:39 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:39.383696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57cbeb-e671-4b48-b71b-b55967f8e88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:39.385024 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57cbeb-e671-4b48-b71b-b55967f8e88f admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:39.385577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57cbeb-e671-4b48-b71b-b55967f8e88f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:39.385872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca6c72-063b-4198-ad0a-b389420020f4 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:39.386291 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57cbeb-e671-4b48-b71b-b55967f8e88f admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:39.387221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57cbeb-e671-4b48-b71b-b55967f8e88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:39.389446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57cbeb-e671-4b48-b71b-b55967f8e88f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:39.415682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 824/1682] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:39.417095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57cbeb-e671-4b48-b71b-b55967f8e88f admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:39.441383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57cbeb-e671-4b48-b71b-b55967f8e88f admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:39.605691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d69f26-63c1-4d62-be8c-11dd606d4b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:39.606455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69f26-63c1-4d62-be8c-11dd606d4b65 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:39.606755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69f26-63c1-4d62-be8c-11dd606d4b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:39.607187 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69f26-63c1-4d62-be8c-11dd606d4b65 admin admin] PATH_INFO: `/v3/projects/3ddb806c902742d1bc37ede524c27140/users/41d49d19fa8442fb855ac0a1926d8587/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:39.608125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca6c72-063b-4198-ad0a-b389420020f4 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:39.621927 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d57a75-35c2-4bda-b484-6a9a777f8032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:39.630059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d57a75-35c2-4bda-b484-6a9a777f8032 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:39.630059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d57a75-35c2-4bda-b484-6a9a777f8032 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:39.630059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d57a75-35c2-4bda-b484-6a9a777f8032 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:39.630059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d57a75-35c2-4bda-b484-6a9a777f8032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:39.632122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d57a75-35c2-4bda-b484-6a9a777f8032 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:39.673117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d69f26-63c1-4d62-be8c-11dd606d4b65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ddb806c902742d1bc37ede524c27140, user_id=41d49d19fa8442fb855ac0a1926d8587, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:39.682948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d69f26-63c1-4d62-be8c-11dd606d4b65 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:39.771715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:39.781021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 859/1683] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:39 2022] POST /identity/v3/users => generated 293 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:39.785852 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d57a75-35c2-4bda-b484-6a9a777f8032 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:39.813372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d57a75-35c2-4bda-b484-6a9a777f8032 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:39.826997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb71f5-0e98-46cf-bc43-31af7108dcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:39.827876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb71f5-0e98-46cf-bc43-31af7108dcde admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:39.830377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb71f5-0e98-46cf-bc43-31af7108dcde admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:39.830377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb71f5-0e98-46cf-bc43-31af7108dcde admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:39.830377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb71f5-0e98-46cf-bc43-31af7108dcde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:39.836146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb71f5-0e98-46cf-bc43-31af7108dcde admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:39.859003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 860/1684] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:26:39.882781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 825/1685] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:39 2022] PUT /identity/v3/projects/3ddb806c902742d1bc37ede524c27140/users/41d49d19fa8442fb855ac0a1926d8587/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:39.883021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb71f5-0e98-46cf-bc43-31af7108dcde admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:39.888022 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a5a15-93a9-4455-a8ae-b08538014507 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:39.890335 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a5a15-93a9-4455-a8ae-b08538014507 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:39.891344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a5a15-93a9-4455-a8ae-b08538014507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:39.936545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb71f5-0e98-46cf-bc43-31af7108dcde admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:39.987846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-f15a5a15-93a9-4455-a8ae-b08538014507 None None] MFA Rules not processed for user `41d49d19fa8442fb855ac0a1926d8587`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:39.992522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f15a5a15-93a9-4455-a8ae-b08538014507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:40.174538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab281a2-897c-467e-8275-fe228d06c4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:40.175198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab281a2-897c-467e-8275-fe228d06c4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:40.175496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab281a2-897c-467e-8275-fe228d06c4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:40.175749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab281a2-897c-467e-8275-fe228d06c4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:40.176415 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab281a2-897c-467e-8275-fe228d06c4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:40.184987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab281a2-897c-467e-8275-fe228d06c4b3 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:40.237148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 861/1686] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:40.246973 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab281a2-897c-467e-8275-fe228d06c4b3 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:40.280201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab281a2-897c-467e-8275-fe228d06c4b3 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:40.294814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a266db-beee-4728-8280-731c49a7c06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:40.295894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a266db-beee-4728-8280-731c49a7c06f admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:40.296361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a266db-beee-4728-8280-731c49a7c06f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:40.296868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a266db-beee-4728-8280-731c49a7c06f admin admin] PATH_INFO: `/v3/projects/ec6c3fb9734148718aa023ef782e5090/users/520e21fdbffa4f7f8ee9519dda5052d7/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:40.337511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a266db-beee-4728-8280-731c49a7c06f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6c3fb9734148718aa023ef782e5090, user_id=520e21fdbffa4f7f8ee9519dda5052d7, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:40.345472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a266db-beee-4728-8280-731c49a7c06f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:40.361337 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 826/1687] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:26:40.370996 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a5a15-93a9-4455-a8ae-b08538014507 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:40.399832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a5a15-93a9-4455-a8ae-b08538014507 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:40.411385 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b5a15-3deb-4f49-a326-e7e46cb09c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:40.412941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b5a15-3deb-4f49-a326-e7e46cb09c53 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:40.413473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b5a15-3deb-4f49-a326-e7e46cb09c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:40.413929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b5a15-3deb-4f49-a326-e7e46cb09c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:40.415333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b5a15-3deb-4f49-a326-e7e46cb09c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:40.420799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b5a15-3deb-4f49-a326-e7e46cb09c53 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:40.533565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 862/1688] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:40 2022] PUT /identity/v3/projects/ec6c3fb9734148718aa023ef782e5090/users/520e21fdbffa4f7f8ee9519dda5052d7/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:40.544169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a266db-beee-4728-8280-731c49a7c06f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:40.547191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a3a266db-beee-4728-8280-731c49a7c06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:40.588680 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 827/1689] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:40 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:40.590556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b2f5c-7375-45c0-a777-b3ee32515746 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:40.591032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b2f5c-7375-45c0-a777-b3ee32515746 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:40.591959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b2f5c-7375-45c0-a777-b3ee32515746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:40.694653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a266db-beee-4728-8280-731c49a7c06f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:40.708295 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-198b2f5c-7375-45c0-a777-b3ee32515746 None None] MFA Rules not processed for user `520e21fdbffa4f7f8ee9519dda5052d7`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:40.713677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-198b2f5c-7375-45c0-a777-b3ee32515746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:40.832543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 828/1690] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:40.832945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b2f5c-7375-45c0-a777-b3ee32515746 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:40.885233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dab4eb-c28a-47e7-9578-adad5f50e286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:40.887314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dab4eb-c28a-47e7-9578-adad5f50e286 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:40.888544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dab4eb-c28a-47e7-9578-adad5f50e286 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:40.888982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dab4eb-c28a-47e7-9578-adad5f50e286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:40.895918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a0dab4eb-c28a-47e7-9578-adad5f50e286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:40.970070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b2f5c-7375-45c0-a777-b3ee32515746 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:41.002606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc01ab7-c3a8-4d63-8891-313a32426738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:41.005084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc01ab7-c3a8-4d63-8891-313a32426738 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:41.007239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc01ab7-c3a8-4d63-8891-313a32426738 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:41.008105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc01ab7-c3a8-4d63-8891-313a32426738 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:41.013824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc01ab7-c3a8-4d63-8891-313a32426738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:41.022634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc01ab7-c3a8-4d63-8891-313a32426738 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:41.035110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dab4eb-c28a-47e7-9578-adad5f50e286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:41.037990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dab4eb-c28a-47e7-9578-adad5f50e286 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:41.060990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 863/1691] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:41.075581 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dab4eb-c28a-47e7-9578-adad5f50e286 service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:41.109022 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dab4eb-c28a-47e7-9578-adad5f50e286 service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:41.123033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da311e30-66c9-4881-8529-cfa675933969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:41.124232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da311e30-66c9-4881-8529-cfa675933969 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:41.124232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da311e30-66c9-4881-8529-cfa675933969 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:41.128527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da311e30-66c9-4881-8529-cfa675933969 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:41.128527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da311e30-66c9-4881-8529-cfa675933969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:41.133555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da311e30-66c9-4881-8529-cfa675933969 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:41.200160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 829/1692] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:40 2022] POST /identity/v3/users => generated 295 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:41.205258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc01ab7-c3a8-4d63-8891-313a32426738 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:41.230508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc01ab7-c3a8-4d63-8891-313a32426738 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:41.235873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b71d8-ee5e-4f03-a890-9375090bb2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:41.236515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b71d8-ee5e-4f03-a890-9375090bb2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:41.236792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b71d8-ee5e-4f03-a890-9375090bb2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:41.237074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b71d8-ee5e-4f03-a890-9375090bb2a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:41.238390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b71d8-ee5e-4f03-a890-9375090bb2a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:41.241302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b71d8-ee5e-4f03-a890-9375090bb2a1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:41.259770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 830/1693] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:41.263695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b71d8-ee5e-4f03-a890-9375090bb2a1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:41.296747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b71d8-ee5e-4f03-a890-9375090bb2a1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:41.313543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8bc7c-1c8e-4f4e-a5ef-ada194f82af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:41.315079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8bc7c-1c8e-4f4e-a5ef-ada194f82af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:41.316420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8bc7c-1c8e-4f4e-a5ef-ada194f82af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:41.318391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8bc7c-1c8e-4f4e-a5ef-ada194f82af4 admin admin] PATH_INFO: `/v3/projects/d8b616ba3c9e426b92f6e41302d6c76c/users/1380d42c1e634f678d1381988c8111c7/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:41.362907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8bc7c-1c8e-4f4e-a5ef-ada194f82af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b616ba3c9e426b92f6e41302d6c76c, user_id=1380d42c1e634f678d1381988c8111c7, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:41.369063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8bc7c-1c8e-4f4e-a5ef-ada194f82af4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:41.403049 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 864/1694] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:41 2022] POST /identity/v3/projects => generated 385 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:41.405309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da311e30-66c9-4881-8529-cfa675933969 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:41.456011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:41.526879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 831/1695] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:41 2022] PUT /identity/v3/projects/d8b616ba3c9e426b92f6e41302d6c76c/users/1380d42c1e634f678d1381988c8111c7/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:41.532338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8bc7c-1c8e-4f4e-a5ef-ada194f82af4 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:41.690740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8bc7c-1c8e-4f4e-a5ef-ada194f82af4 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:41.700952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045679-1097-44a6-8810-6a0ce495d83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:41.701641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045679-1097-44a6-8810-6a0ce495d83a admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:41.701920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045679-1097-44a6-8810-6a0ce495d83a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:41.702226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045679-1097-44a6-8810-6a0ce495d83a admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:41.703011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae045679-1097-44a6-8810-6a0ce495d83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:41.709177 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae045679-1097-44a6-8810-6a0ce495d83a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:41.739658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 832/1696] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:41.746293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045679-1097-44a6-8810-6a0ce495d83a admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:41.756736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da311e30-66c9-4881-8529-cfa675933969 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:41.772982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045679-1097-44a6-8810-6a0ce495d83a admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:41.788301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f0c78-4a76-4f2e-861a-ef57ab527677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:41.789332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f0c78-4a76-4f2e-861a-ef57ab527677 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:41.789332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f0c78-4a76-4f2e-861a-ef57ab527677 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:41.789599 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f0c78-4a76-4f2e-861a-ef57ab527677 admin admin] PATH_INFO: `/v3/projects/d8b616ba3c9e426b92f6e41302d6c76c/users/1380d42c1e634f678d1381988c8111c7/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:41.791672 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f0c78-4a76-4f2e-861a-ef57ab527677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b616ba3c9e426b92f6e41302d6c76c, user_id=1380d42c1e634f678d1381988c8111c7, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:41.793169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a080a-2d2e-40ad-b80b-62f57433c274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:41.794238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a080a-2d2e-40ad-b80b-62f57433c274 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:41.794406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f0c78-4a76-4f2e-861a-ef57ab527677 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:41.794784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a080a-2d2e-40ad-b80b-62f57433c274 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:41.796221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a080a-2d2e-40ad-b80b-62f57433c274 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:41.801478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a080a-2d2e-40ad-b80b-62f57433c274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:41.805882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a080a-2d2e-40ad-b80b-62f57433c274 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:41.834576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 833/1697] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:41 2022] PUT /identity/v3/projects/d8b616ba3c9e426b92f6e41302d6c76c/users/1380d42c1e634f678d1381988c8111c7/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:26:41.844239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e1095-e7f6-4243-81ac-e7a14a5d4e33 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:41.844553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e1095-e7f6-4243-81ac-e7a14a5d4e33 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:41.844839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e1095-e7f6-4243-81ac-e7a14a5d4e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:41.901953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ab5e1095-e7f6-4243-81ac-e7a14a5d4e33 None None] MFA Rules not processed for user `1380d42c1e634f678d1381988c8111c7`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:41.903258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ab5e1095-e7f6-4243-81ac-e7a14a5d4e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:41.981637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 865/1698] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:41 2022] POST /identity/v3/users => generated 299 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:41.988306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a080a-2d2e-40ad-b80b-62f57433c274 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:42.077436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a080a-2d2e-40ad-b80b-62f57433c274 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:42.127287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 834/1699] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:42.261535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e1095-e7f6-4243-81ac-e7a14a5d4e33 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:42.267322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ab5e1095-e7f6-4243-81ac-e7a14a5d4e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:42.379192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939581c-964d-4f93-b6a6-9cd8bdd2b36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:42.382210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939581c-964d-4f93-b6a6-9cd8bdd2b36d admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:42.382748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939581c-964d-4f93-b6a6-9cd8bdd2b36d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:42.384091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939581c-964d-4f93-b6a6-9cd8bdd2b36d admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:42.388335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939581c-964d-4f93-b6a6-9cd8bdd2b36d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:42.392889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939581c-964d-4f93-b6a6-9cd8bdd2b36d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:42.422460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 866/1700] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:26:42.430915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939581c-964d-4f93-b6a6-9cd8bdd2b36d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:42.456320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:42.462900 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939581c-964d-4f93-b6a6-9cd8bdd2b36d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:42.472416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac37345-8b4b-4da8-a13f-938e473d0ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:42.474456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac37345-8b4b-4da8-a13f-938e473d0ffa admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:42.475484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac37345-8b4b-4da8-a13f-938e473d0ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:42.476708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac37345-8b4b-4da8-a13f-938e473d0ffa admin admin] PATH_INFO: `/v3/projects/37ced8b9f49b4761bc502314409e9583/users/60d33790f00b413fa48f6ac54922c215/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:42.529834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac37345-8b4b-4da8-a13f-938e473d0ffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ced8b9f49b4761bc502314409e9583, user_id=60d33790f00b413fa48f6ac54922c215, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:42.548749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac37345-8b4b-4da8-a13f-938e473d0ffa admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:42.575720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e1095-e7f6-4243-81ac-e7a14a5d4e33 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:42.591301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adcdd8-6d27-47d6-a78a-378aa2ee4512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:42.594454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adcdd8-6d27-47d6-a78a-378aa2ee4512 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:42.594454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adcdd8-6d27-47d6-a78a-378aa2ee4512 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:42.594454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adcdd8-6d27-47d6-a78a-378aa2ee4512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:42.684734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 867/1701] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:42 2022] PUT /identity/v3/projects/37ced8b9f49b4761bc502314409e9583/users/60d33790f00b413fa48f6ac54922c215/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:42.691069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac37345-8b4b-4da8-a13f-938e473d0ffa admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:42.839413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adcdd8-6d27-47d6-a78a-378aa2ee4512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:42.843550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adcdd8-6d27-47d6-a78a-378aa2ee4512 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:42.858201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac37345-8b4b-4da8-a13f-938e473d0ffa admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:42.868221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4ab73-38b2-484e-afe7-d0e5c260adc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:42.869221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ab73-38b2-484e-afe7-d0e5c260adc2 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:42.869821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ab73-38b2-484e-afe7-d0e5c260adc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:42.870273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ab73-38b2-484e-afe7-d0e5c260adc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:42.872221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 835/1702] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:42.876899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4ab73-38b2-484e-afe7-d0e5c260adc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:42.882312 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4ab73-38b2-484e-afe7-d0e5c260adc2 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:42.917588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adcdd8-6d27-47d6-a78a-378aa2ee4512 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:42.919696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 868/1703] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:42.934902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adcdd8-6d27-47d6-a78a-378aa2ee4512 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:42.948429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad05207-98bb-4361-982b-eb8036c0cc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:42.949773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad05207-98bb-4361-982b-eb8036c0cc79 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:42.950554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad05207-98bb-4361-982b-eb8036c0cc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:42.951375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad05207-98bb-4361-982b-eb8036c0cc79 admin admin] PATH_INFO: `/v3/projects/37ced8b9f49b4761bc502314409e9583/users/60d33790f00b413fa48f6ac54922c215/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:42.957973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad05207-98bb-4361-982b-eb8036c0cc79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ced8b9f49b4761bc502314409e9583, user_id=60d33790f00b413fa48f6ac54922c215, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:42.964071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad05207-98bb-4361-982b-eb8036c0cc79 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:42.995193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 836/1704] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:42 2022] PUT /identity/v3/projects/37ced8b9f49b4761bc502314409e9583/users/60d33790f00b413fa48f6ac54922c215/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:43.007561 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c3e29-9f48-4e8a-bd13-67cf5c2ccacd None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:43.008072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c3e29-9f48-4e8a-bd13-67cf5c2ccacd None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:43.008548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c3e29-9f48-4e8a-bd13-67cf5c2ccacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:43.106204 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f40c3e29-9f48-4e8a-bd13-67cf5c2ccacd None None] MFA Rules not processed for user `60d33790f00b413fa48f6ac54922c215`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:43.107723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f40c3e29-9f48-4e8a-bd13-67cf5c2ccacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:43.285862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad05207-98bb-4361-982b-eb8036c0cc79 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:43.289064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bad05207-98bb-4361-982b-eb8036c0cc79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:43.304443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 869/1705] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:43.467780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c3e29-9f48-4e8a-bd13-67cf5c2ccacd None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:43.570047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad05207-98bb-4361-982b-eb8036c0cc79 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:43.594724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:43.595664 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:43.595996 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:43.596452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] PATH_INFO: `/v3/users/9e88f01e082a4ebbad83de0e02de82b8` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:43.598390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e88f01e082a4ebbad83de0e02de82b8)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:43.609169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:43.698807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c3e29-9f48-4e8a-bd13-67cf5c2ccacd None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:43.704388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab81b5-b52f-4783-834c-f46b19c287ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:43.705240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab81b5-b52f-4783-834c-f46b19c287ad service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:43.705524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab81b5-b52f-4783-834c-f46b19c287ad service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:43.705796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab81b5-b52f-4783-834c-f46b19c287ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:43.721065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab81b5-b52f-4783-834c-f46b19c287ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:43.723302 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab81b5-b52f-4783-834c-f46b19c287ad service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:43.740844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 870/1706] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:43.950160 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e88f01e082a4ebbad83de0e02de82b8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:43.950954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e88f01e082a4ebbad83de0e02de82b8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:43.951581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e88f01e082a4ebbad83de0e02de82b8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:43.972962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e88f01e082a4ebbad83de0e02de82b8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:44.005085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e88f01e082a4ebbad83de0e02de82b8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:44.076010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:44.110949 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 837/1707] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:43 2022] DELETE /identity/v3/users/9e88f01e082a4ebbad83de0e02de82b8 => generated 0 bytes in 826 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:44.118241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8823c-1e07-4c91-92d9-699b9a6f5ff9 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:44.118774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8823c-1e07-4c91-92d9-699b9a6f5ff9 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:44.118872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8823c-1e07-4c91-92d9-699b9a6f5ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:44.127634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:44.189332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:44.191580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-b9b8823c-1e07-4c91-92d9-699b9a6f5ff9 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:44.192219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:44.198703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b9b8823c-1e07-4c91-92d9-699b9a6f5ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:44.480815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 871/1708] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:44.570294 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa7b64-d822-4583-8d91-97205eabdb73 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:44.579764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:44.580251 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:44.580501 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:44.581347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] PATH_INFO: `/v3/users/520e21fdbffa4f7f8ee9519dda5052d7` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:44.582891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=520e21fdbffa4f7f8ee9519dda5052d7)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:44.585819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:44.700091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df8c9-0776-44c7-a3ab-4fe66fcce500 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:44.700517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df8c9-0776-44c7-a3ab-4fe66fcce500 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:44.700987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df8c9-0776-44c7-a3ab-4fe66fcce500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:44.851505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-680df8c9-0776-44c7-a3ab-4fe66fcce500 None None] MFA Rules not processed for user `864d7f47381e4e4498ad110dbfcfc154`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:44.861164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-680df8c9-0776-44c7-a3ab-4fe66fcce500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:45.084866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '520e21fdbffa4f7f8ee9519dda5052d7'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:45.084866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '520e21fdbffa4f7f8ee9519dda5052d7'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:45.084866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '520e21fdbffa4f7f8ee9519dda5052d7'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:45.119337 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '520e21fdbffa4f7f8ee9519dda5052d7'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:45.164172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '520e21fdbffa4f7f8ee9519dda5052d7'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:45.200332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 872/1709] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:26:45.213777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680df8c9-0776-44c7-a3ab-4fe66fcce500 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:45.224047 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-680df8c9-0776-44c7-a3ab-4fe66fcce500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:45.330824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 838/1710] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:44 2022] DELETE /identity/v3/users/520e21fdbffa4f7f8ee9519dda5052d7 => generated 0 bytes in 1142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:45.331983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:45.338241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:45.500454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680df8c9-0776-44c7-a3ab-4fe66fcce500 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:45.621703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32110c-f4fc-450f-a6af-523a5ef43c91 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:45.637600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054577c-46da-4071-af00-d6921d5e7c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:45.638651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054577c-46da-4071-af00-d6921d5e7c64 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:45.639065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054577c-46da-4071-af00-d6921d5e7c64 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:45.639966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054577c-46da-4071-af00-d6921d5e7c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:45.717317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:45.777838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:45.779341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb798a14-32a6-4d84-b49b-a567e67954e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:45.779994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb798a14-32a6-4d84-b49b-a567e67954e2 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:45.780279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb798a14-32a6-4d84-b49b-a567e67954e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:45.780553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb798a14-32a6-4d84-b49b-a567e67954e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:45.789744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fb798a14-32a6-4d84-b49b-a567e67954e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:45.804522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054577c-46da-4071-af00-d6921d5e7c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:45.806551 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054577c-46da-4071-af00-d6921d5e7c64 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:45.819677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 839/1711] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:45.821130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda0a1c-488a-4fd0-8720-f031074d4f3b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:45.821525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda0a1c-488a-4fd0-8720-f031074d4f3b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:45.823667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda0a1c-488a-4fd0-8720-f031074d4f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:45.871087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2eda0a1c-488a-4fd0-8720-f031074d4f3b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:45.875157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2eda0a1c-488a-4fd0-8720-f031074d4f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:46.046331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 840/1712] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:46.061804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda0a1c-488a-4fd0-8720-f031074d4f3b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:46.063408 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2eda0a1c-488a-4fd0-8720-f031074d4f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:46.085027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda0a1c-488a-4fd0-8720-f031074d4f3b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:46.091237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a1f58-b36b-4fa7-af38-59d51e1398fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:46.091922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a1f58-b36b-4fa7-af38-59d51e1398fe service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:46.092206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a1f58-b36b-4fa7-af38-59d51e1398fe service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:46.092515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a1f58-b36b-4fa7-af38-59d51e1398fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:46.121607 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a1f58-b36b-4fa7-af38-59d51e1398fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:46.123674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a1f58-b36b-4fa7-af38-59d51e1398fe service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:46.146759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 841/1713] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:46.201401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb798a14-32a6-4d84-b49b-a567e67954e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:46.203160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb798a14-32a6-4d84-b49b-a567e67954e2 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:46.267497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 873/1714] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 1066 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:46.299964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:46.877198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:47.692392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ffa96-82a1-4f82-b843-e15e2c12c4ad None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:47.692852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ffa96-82a1-4f82-b843-e15e2c12c4ad None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:47.694143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ffa96-82a1-4f82-b843-e15e2c12c4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:47.751056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-f45ffa96-82a1-4f82-b843-e15e2c12c4ad None None] MFA Rules not processed for user `f46cbc7de18c4f9fb6f54e96be53fea0`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:47.754519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f45ffa96-82a1-4f82-b843-e15e2c12c4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:47.952620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 842/1715] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:47.964542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324f6ec-a101-4321-8b9d-823567e151b6 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:47.965329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324f6ec-a101-4321-8b9d-823567e151b6 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:47.965795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324f6ec-a101-4321-8b9d-823567e151b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:48.064447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:48.071467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-b324f6ec-a101-4321-8b9d-823567e151b6 None None] MFA Rules not processed for user `f46cbc7de18c4f9fb6f54e96be53fea0`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:48.078169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b324f6ec-a101-4321-8b9d-823567e151b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:48.139117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 874/1716] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:48.161440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ffa96-82a1-4f82-b843-e15e2c12c4ad None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:48.164979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f45ffa96-82a1-4f82-b843-e15e2c12c4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:48.268422 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ffa96-82a1-4f82-b843-e15e2c12c4ad None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:48.301033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743c0f3-f51c-4a97-8102-ceb16f4c8cb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:48.302251 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743c0f3-f51c-4a97-8102-ceb16f4c8cb9 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:48.303215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743c0f3-f51c-4a97-8102-ceb16f4c8cb9 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:48.303802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743c0f3-f51c-4a97-8102-ceb16f4c8cb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:48.348763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743c0f3-f51c-4a97-8102-ceb16f4c8cb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:48.352297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743c0f3-f51c-4a97-8102-ceb16f4c8cb9 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:48.372141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324f6ec-a101-4321-8b9d-823567e151b6 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:48.374526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b324f6ec-a101-4321-8b9d-823567e151b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:48.390729 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 843/1717] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:48.432745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324f6ec-a101-4321-8b9d-823567e151b6 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:48.452495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:48.453248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:48.453860 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:48.454548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] PATH_INFO: `/v3/users/dcaea48773d2426eacacdcab56c1669b` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:48.457892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcaea48773d2426eacacdcab56c1669b)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:48.461096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: WARNING py.warnings [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 17:26:48.461096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: warnings.warn(msg) Aug 22 17:26:48.461096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: Aug 22 17:26:48.464789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:48.472753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743c0f3-f51c-4a97-8102-ceb16f4c8cb9 service nova] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:48.478721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0743c0f3-f51c-4a97-8102-ceb16f4c8cb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:48.581262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743c0f3-f51c-4a97-8102-ceb16f4c8cb9 service nova] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:48.592278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:48.593529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:48.594127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:48.594758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] PATH_INFO: `/v3/projects/ec6c3fb9734148718aa023ef782e5090` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:48.604163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6c3fb9734148718aa023ef782e5090)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:48.609673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:49.036148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcaea48773d2426eacacdcab56c1669b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.039928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcaea48773d2426eacacdcab56c1669b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.047191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcaea48773d2426eacacdcab56c1669b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.057054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6c3fb9734148718aa023ef782e5090', 'request_id': 'req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.087866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6c3fb9734148718aa023ef782e5090', 'request_id': 'req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.160317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcaea48773d2426eacacdcab56c1669b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.202728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6c3fb9734148718aa023ef782e5090', 'request_id': 'req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.227315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-461b033d-7573-4098-a14d-07f44b8671da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcaea48773d2426eacacdcab56c1669b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.239093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] The token cache is being invalidate because project ec6c3fb9734148718aa023ef782e5090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:26:49.239892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.243825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.256391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 844/1718] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:26:48 2022] DELETE /identity/v3/projects/ec6c3fb9734148718aa023ef782e5090 => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:49.260275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:49.265220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:49.351217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 875/1719] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:48 2022] DELETE /identity/v3/users/dcaea48773d2426eacacdcab56c1669b => generated 0 bytes in 975 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:49.359019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad864d-c9d1-479b-b51b-3ff0fd6274c6 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:49.360903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad864d-c9d1-479b-b51b-3ff0fd6274c6 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:49.362438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad864d-c9d1-479b-b51b-3ff0fd6274c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:49.447499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-24ad864d-c9d1-479b-b51b-3ff0fd6274c6 None None] MFA Rules not processed for user `b0db362075b04b2294a6268576a5bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:49.456170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-24ad864d-c9d1-479b-b51b-3ff0fd6274c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:49.466791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871efbbc-aaa1-48cc-9fd3-a065dcde3d12 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:49.524915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:49.617316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:49.618003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:49.618401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:49.618714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] PATH_INFO: `/v3/projects/e6a0059a8ba14a1aaab08cccc86fe6c6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:49.626239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a0059a8ba14a1aaab08cccc86fe6c6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:49.628397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:49.630367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 876/1720] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:49.632616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad864d-c9d1-479b-b51b-3ff0fd6274c6 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:49.652273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-24ad864d-c9d1-479b-b51b-3ff0fd6274c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:49.710071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad864d-c9d1-479b-b51b-3ff0fd6274c6 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:49.730143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:49.731716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:49.732142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:49.732570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] PATH_INFO: `/v3/users/60d33790f00b413fa48f6ac54922c215` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:49.739953 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] RBAC: Authorizing `identity:delete_user(user_id=60d33790f00b413fa48f6ac54922c215)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:49.746115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:49.841337 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a0059a8ba14a1aaab08cccc86fe6c6', 'request_id': 'req-f3d469ca-25f0-41b7-99c5-088b385f898c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:49.873088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a0059a8ba14a1aaab08cccc86fe6c6', 'request_id': 'req-f3d469ca-25f0-41b7-99c5-088b385f898c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:50.009323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a0059a8ba14a1aaab08cccc86fe6c6', 'request_id': 'req-f3d469ca-25f0-41b7-99c5-088b385f898c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:50.051556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:50.059408 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] The token cache is being invalidate because project e6a0059a8ba14a1aaab08cccc86fe6c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:26:50.060138 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:50.063926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3d469ca-25f0-41b7-99c5-088b385f898c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:50.078707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 845/1721] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:26:49 2022] DELETE /identity/v3/projects/e6a0059a8ba14a1aaab08cccc86fe6c6 => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:50.078707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c9f81-9b3a-479d-bb44-93ca546f7bdc None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:50.079045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c9f81-9b3a-479d-bb44-93ca546f7bdc None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:50.079550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c9f81-9b3a-479d-bb44-93ca546f7bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:50.138255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60d33790f00b413fa48f6ac54922c215'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:50.144247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60d33790f00b413fa48f6ac54922c215'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:50.155751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60d33790f00b413fa48f6ac54922c215'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:50.176934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-e33c9f81-9b3a-479d-bb44-93ca546f7bdc None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:50.180890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e33c9f81-9b3a-479d-bb44-93ca546f7bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:50.277408 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60d33790f00b413fa48f6ac54922c215'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:50.385610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-242c0f85-83ce-4ab5-b7cc-d811970e447f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60d33790f00b413fa48f6ac54922c215'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:50.410413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:50.489583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 846/1722] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:50.493293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f12f75-0b11-4d82-b219-ac762aeccd2a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:50.504206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 877/1723] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:49 2022] DELETE /identity/v3/users/60d33790f00b413fa48f6ac54922c215 => generated 0 bytes in 873 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:50.504937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f12f75-0b11-4d82-b219-ac762aeccd2a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:50.505663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f12f75-0b11-4d82-b219-ac762aeccd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:50.511189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf62a0a-5b06-4304-9021-bf7ff0ebe126 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:50.511715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf62a0a-5b06-4304-9021-bf7ff0ebe126 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:50.514821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf62a0a-5b06-4304-9021-bf7ff0ebe126 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:50.519527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 878/1724] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 22 17:26:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:26:50.527393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68df02-379b-4ab3-bd42-bfb74fbc2621 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:50.527955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68df02-379b-4ab3-bd42-bfb74fbc2621 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:50.528829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68df02-379b-4ab3-bd42-bfb74fbc2621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:50.600878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-73f12f75-0b11-4d82-b219-ac762aeccd2a None None] MFA Rules not processed for user `9eedc633493b478a9009f7bce07696a8`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:50.602527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-73f12f75-0b11-4d82-b219-ac762aeccd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:50.611492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-af68df02-379b-4ab3-bd42-bfb74fbc2621 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:50.615745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-af68df02-379b-4ab3-bd42-bfb74fbc2621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:50.854764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 847/1725] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:50.859404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed6942-f7c9-437f-9a7d-8525d2290b8c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:50.859919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed6942-f7c9-437f-9a7d-8525d2290b8c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:50.860532 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed6942-f7c9-437f-9a7d-8525d2290b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:50.939123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 879/1726] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:50.959014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68df02-379b-4ab3-bd42-bfb74fbc2621 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:50.972293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-af68df02-379b-4ab3-bd42-bfb74fbc2621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:51.010759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1aed6942-f7c9-437f-9a7d-8525d2290b8c None None] MFA Rules not processed for user `c29a91e099ba4cd5886fd910d978fb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:51.012782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1aed6942-f7c9-437f-9a7d-8525d2290b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:51.211417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 848/1727] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:26:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:51.215538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e79fae-9732-4dba-a7a1-2848bb7d78f1 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:51.216651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e79fae-9732-4dba-a7a1-2848bb7d78f1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:51.217513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e79fae-9732-4dba-a7a1-2848bb7d78f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:51.234523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68df02-379b-4ab3-bd42-bfb74fbc2621 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:51.243345 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3867d21-16a6-4dff-99ea-350806919eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:51.245220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3867d21-16a6-4dff-99ea-350806919eaa service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:51.245847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3867d21-16a6-4dff-99ea-350806919eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:51.246563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3867d21-16a6-4dff-99ea-350806919eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:51.283268 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d5e79fae-9732-4dba-a7a1-2848bb7d78f1 None None] MFA Rules not processed for user `9eedc633493b478a9009f7bce07696a8`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:51.285574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d5e79fae-9732-4dba-a7a1-2848bb7d78f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:51.289838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3867d21-16a6-4dff-99ea-350806919eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:51.294405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:51.297422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3867d21-16a6-4dff-99ea-350806919eaa service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:51.319543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 849/1728] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:51.326460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad40000-b0ec-4584-9bbc-9e5db857d2bc None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:51.326925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad40000-b0ec-4584-9bbc-9e5db857d2bc None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:51.327290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad40000-b0ec-4584-9bbc-9e5db857d2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:51.342536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 880/1729] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:51.345474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51ed23-11e8-403c-b992-bc6dd1368e9d None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:51.346369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51ed23-11e8-403c-b992-bc6dd1368e9d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:51.358430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51ed23-11e8-403c-b992-bc6dd1368e9d None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:51.361016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-dad40000-b0ec-4584-9bbc-9e5db857d2bc None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:51.363977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 881/1730] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 22 17:26:51 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:26:51.367464 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51ed23-11e8-403c-b992-bc6dd1368e9d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:51.372801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-dad40000-b0ec-4584-9bbc-9e5db857d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:51.393661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 850/1731] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:51.393661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e01da7-283f-4a74-8c72-c14e125b0187 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:51.393661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e01da7-283f-4a74-8c72-c14e125b0187 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:51.393661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e01da7-283f-4a74-8c72-c14e125b0187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:51.403487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51ed23-11e8-403c-b992-bc6dd1368e9d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:51.412549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1b38e-e403-4717-ac0b-e0f6e88957e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:51.419579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1b38e-e403-4717-ac0b-e0f6e88957e6 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:51.419579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1b38e-e403-4717-ac0b-e0f6e88957e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:51.419579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1b38e-e403-4717-ac0b-e0f6e88957e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:51.419961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-81e01da7-283f-4a74-8c72-c14e125b0187 None None] MFA Rules not processed for user `c29a91e099ba4cd5886fd910d978fb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:51.419961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-81e01da7-283f-4a74-8c72-c14e125b0187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:51.441923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1b38e-e403-4717-ac0b-e0f6e88957e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:51.446075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1b38e-e403-4717-ac0b-e0f6e88957e6 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:51.447754 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 851/1732] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:26:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:51.456261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e01da7-283f-4a74-8c72-c14e125b0187 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:51.465805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 882/1733] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:51.472833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e01da7-283f-4a74-8c72-c14e125b0187 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:51.479131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63bc4-955c-48f1-bb7e-8e2ba0d29f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:51.479944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63bc4-955c-48f1-bb7e-8e2ba0d29f10 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:51.480356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63bc4-955c-48f1-bb7e-8e2ba0d29f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:51.481222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63bc4-955c-48f1-bb7e-8e2ba0d29f10 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:51.482276 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63bc4-955c-48f1-bb7e-8e2ba0d29f10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:51.482747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63bc4-955c-48f1-bb7e-8e2ba0d29f10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:51.484652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63bc4-955c-48f1-bb7e-8e2ba0d29f10 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:51.501516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1b38e-e403-4717-ac0b-e0f6e88957e6 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:51.505355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-41d1b38e-e403-4717-ac0b-e0f6e88957e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:51.531850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1b38e-e403-4717-ac0b-e0f6e88957e6 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:51.550617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae24cb9-876a-49e3-a6ba-5e18f13eddf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:51.551949 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae24cb9-876a-49e3-a6ba-5e18f13eddf0 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:51.553830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae24cb9-876a-49e3-a6ba-5e18f13eddf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:51.556066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae24cb9-876a-49e3-a6ba-5e18f13eddf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:51.559691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 852/1734] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:51.566640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63bc4-955c-48f1-bb7e-8e2ba0d29f10 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:51.597510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae24cb9-876a-49e3-a6ba-5e18f13eddf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:51.604688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63bc4-955c-48f1-bb7e-8e2ba0d29f10 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:51.616041 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae24cb9-876a-49e3-a6ba-5e18f13eddf0 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:51.628402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d27fed-32a5-408e-b4fe-7c0d6f62f50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:51.632218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d27fed-32a5-408e-b4fe-7c0d6f62f50b admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:51.632734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d27fed-32a5-408e-b4fe-7c0d6f62f50b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:51.633224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d27fed-32a5-408e-b4fe-7c0d6f62f50b admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:51.633976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 883/1735] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:51.634447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d27fed-32a5-408e-b4fe-7c0d6f62f50b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:51.655834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d27fed-32a5-408e-b4fe-7c0d6f62f50b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:51.772307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae24cb9-876a-49e3-a6ba-5e18f13eddf0 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:51.775119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9ae24cb9-876a-49e3-a6ba-5e18f13eddf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:51.805591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae24cb9-876a-49e3-a6ba-5e18f13eddf0 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:51.816951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 853/1736] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:51 2022] POST /identity/v3/projects => generated 375 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:26:51.823982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d27fed-32a5-408e-b4fe-7c0d6f62f50b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:51.830886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:51.831802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:51.833466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:51.835472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] PATH_INFO: `/v3/projects/37ced8b9f49b4761bc502314409e9583` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:51.839382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ced8b9f49b4761bc502314409e9583)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:51.844274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:51.932271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:52.056723 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d27fed-32a5-408e-b4fe-7c0d6f62f50b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:52.058990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ced8b9f49b4761bc502314409e9583', 'request_id': 'req-8dc2d47c-0340-41f1-adc1-dd03be10e01e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:52.077703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ced8b9f49b4761bc502314409e9583', 'request_id': 'req-8dc2d47c-0340-41f1-adc1-dd03be10e01e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:52.124048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ced8b9f49b4761bc502314409e9583', 'request_id': 'req-8dc2d47c-0340-41f1-adc1-dd03be10e01e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:52.256340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] The token cache is being invalidate because project 37ced8b9f49b4761bc502314409e9583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:26:52.258672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:52.263150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8dc2d47c-0340-41f1-adc1-dd03be10e01e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:52.279611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 884/1737] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:26:51 2022] DELETE /identity/v3/projects/37ced8b9f49b4761bc502314409e9583 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:26:52.285698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8464c-d907-4802-8513-00656774aa14 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:52.291636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8464c-d907-4802-8513-00656774aa14 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:52.292793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8464c-d907-4802-8513-00656774aa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:52.316349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe032f0-ce71-4234-aed8-cba4348f4bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:52.317713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe032f0-ce71-4234-aed8-cba4348f4bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:52.319511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe032f0-ce71-4234-aed8-cba4348f4bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:52.323019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe032f0-ce71-4234-aed8-cba4348f4bf5 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:52.326188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe032f0-ce71-4234-aed8-cba4348f4bf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:52.336656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe032f0-ce71-4234-aed8-cba4348f4bf5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:52.372446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-fdd8464c-d907-4802-8513-00656774aa14 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:52.376517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fdd8464c-d907-4802-8513-00656774aa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:52.426213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 885/1738] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:52.437886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5fb07-d11f-4c0d-887b-dd0c6fba8003 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:52.438443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5fb07-d11f-4c0d-887b-dd0c6fba8003 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:52.438992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5fb07-d11f-4c0d-887b-dd0c6fba8003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:52.468688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 854/1739] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:51 2022] POST /identity/v3/users => generated 288 bytes in 645 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:52.474112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe032f0-ce71-4234-aed8-cba4348f4bf5 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:52.480222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-dbe032f0-ce71-4234-aed8-cba4348f4bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:52.505522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe032f0-ce71-4234-aed8-cba4348f4bf5 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:52.523936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37a38e-aa72-4e9f-a2e2-219e8f5d6a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:52.524586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a38e-aa72-4e9f-a2e2-219e8f5d6a10 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:52.524871 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a38e-aa72-4e9f-a2e2-219e8f5d6a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:52.537748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a38e-aa72-4e9f-a2e2-219e8f5d6a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:52.537748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37a38e-aa72-4e9f-a2e2-219e8f5d6a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:52.537748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37a38e-aa72-4e9f-a2e2-219e8f5d6a10 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:52.538613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c9d5fb07-d11f-4c0d-887b-dd0c6fba8003 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:52.538613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c9d5fb07-d11f-4c0d-887b-dd0c6fba8003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:52.556082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 886/1740] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:52.562553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 855/1741] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:52.563340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5fb07-d11f-4c0d-887b-dd0c6fba8003 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:52.563518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37a38e-aa72-4e9f-a2e2-219e8f5d6a10 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:52.564757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c9d5fb07-d11f-4c0d-887b-dd0c6fba8003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:52.585512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37a38e-aa72-4e9f-a2e2-219e8f5d6a10 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:52.646423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447feb8-2357-468e-8555-e0120b16f26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:52.648472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447feb8-2357-468e-8555-e0120b16f26b admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:52.652051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447feb8-2357-468e-8555-e0120b16f26b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:52.652051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447feb8-2357-468e-8555-e0120b16f26b admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:52.652051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447feb8-2357-468e-8555-e0120b16f26b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:26:52.652051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447feb8-2357-468e-8555-e0120b16f26b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:52.653718 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447feb8-2357-468e-8555-e0120b16f26b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:52.685214 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 856/1742] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:26:52.685214 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447feb8-2357-468e-8555-e0120b16f26b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:52.736321 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447feb8-2357-468e-8555-e0120b16f26b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:52.755291 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bec856-4a63-458b-ba57-eefd88a86f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:52.761053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bec856-4a63-458b-ba57-eefd88a86f31 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:52.761053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bec856-4a63-458b-ba57-eefd88a86f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:52.761053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bec856-4a63-458b-ba57-eefd88a86f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:52.761053 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bec856-4a63-458b-ba57-eefd88a86f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:52.763646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bec856-4a63-458b-ba57-eefd88a86f31 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:52.788214 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 857/1743] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:52.789098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bec856-4a63-458b-ba57-eefd88a86f31 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:52.789237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5fb07-d11f-4c0d-887b-dd0c6fba8003 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:52.800431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4ef2a-f114-450e-bfa2-124b4325eb0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:52.801289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4ef2a-f114-450e-bfa2-124b4325eb0c service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:52.801631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4ef2a-f114-450e-bfa2-124b4325eb0c service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:52.801969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4ef2a-f114-450e-bfa2-124b4325eb0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:52.804407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c2d4ef2a-f114-450e-bfa2-124b4325eb0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:52.816071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bec856-4a63-458b-ba57-eefd88a86f31 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:52.829745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ab230-52c8-4dca-af3d-90e3a2d9818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:52.830389 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ab230-52c8-4dca-af3d-90e3a2d9818a admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:52.830695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ab230-52c8-4dca-af3d-90e3a2d9818a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:52.830970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ab230-52c8-4dca-af3d-90e3a2d9818a admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:52.831623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ab230-52c8-4dca-af3d-90e3a2d9818a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:52.836103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ab230-52c8-4dca-af3d-90e3a2d9818a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:52.945564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4ef2a-f114-450e-bfa2-124b4325eb0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:52.977783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4ef2a-f114-450e-bfa2-124b4325eb0c service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:52.986338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 858/1744] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:52 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:52.989594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e9f64-fb04-4d21-ba9f-25b68aac7bde None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:52.992271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e9f64-fb04-4d21-ba9f-25b68aac7bde None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:52.993288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e9f64-fb04-4d21-ba9f-25b68aac7bde None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:52.998565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 859/1745] 10.222.0.20 () {54 vars in 961 bytes} [Mon Aug 22 17:26:52 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:26:53.001412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e9f64-fb04-4d21-ba9f-25b68aac7bde None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:53.036476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 887/1746] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:26:53.040029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4ef2a-f114-450e-bfa2-124b4325eb0c service nova] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:53.054856 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4ef2a-f114-450e-bfa2-124b4325eb0c service nova] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:53.220694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e9f64-fb04-4d21-ba9f-25b68aac7bde None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:53.226617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa86d8a-994e-40b6-b37f-9ae78c3a627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:53.227319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa86d8a-994e-40b6-b37f-9ae78c3a627f admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:53.227601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa86d8a-994e-40b6-b37f-9ae78c3a627f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:53.227874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa86d8a-994e-40b6-b37f-9ae78c3a627f admin admin] PATH_INFO: `/v3/projects/0179c56cebe24f9fbe5d907c375f6ab0/users/00aefa0efeb54dcb9e21dc1b8a9a26d8/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:53.269097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa86d8a-994e-40b6-b37f-9ae78c3a627f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0179c56cebe24f9fbe5d907c375f6ab0, user_id=00aefa0efeb54dcb9e21dc1b8a9a26d8, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:53.272934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa86d8a-994e-40b6-b37f-9ae78c3a627f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:53.293045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc0d2d-f99c-4a36-aa40-af97b1e82024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:53.293882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc0d2d-f99c-4a36-aa40-af97b1e82024 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:53.294263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc0d2d-f99c-4a36-aa40-af97b1e82024 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:53.294984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc0d2d-f99c-4a36-aa40-af97b1e82024 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:53.297554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc0d2d-f99c-4a36-aa40-af97b1e82024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:53.301610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc0d2d-f99c-4a36-aa40-af97b1e82024 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:53.373646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 860/1747] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:52 2022] PUT /identity/v3/projects/0179c56cebe24f9fbe5d907c375f6ab0/users/00aefa0efeb54dcb9e21dc1b8a9a26d8/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:53.375843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24039b82-98e3-4079-b9ee-f00f7d47ca4b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:53.376272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24039b82-98e3-4079-b9ee-f00f7d47ca4b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:53.376692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24039b82-98e3-4079-b9ee-f00f7d47ca4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:53.424293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-24039b82-98e3-4079-b9ee-f00f7d47ca4b None None] MFA Rules not processed for user `26b5021285c44db9b05f9d15ceaf797f`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:53.425686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-24039b82-98e3-4079-b9ee-f00f7d47ca4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:53.487215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 888/1748] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:53 2022] POST /identity/v3/users => generated 291 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:53.488776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05da12-c571-4a41-815d-78b3c958ed90 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:53.489166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05da12-c571-4a41-815d-78b3c958ed90 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:53.489544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05da12-c571-4a41-815d-78b3c958ed90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:53.576012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1a05da12-c571-4a41-815d-78b3c958ed90 None None] MFA Rules not processed for user `864d7f47381e4e4498ad110dbfcfc154`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:53.582723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1a05da12-c571-4a41-815d-78b3c958ed90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:53.625480 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:53.629929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 861/1749] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Aug 22 17:26:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:53.640461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be22b5-8b0f-4911-a28a-17fbdc5b3288 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:53.640906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be22b5-8b0f-4911-a28a-17fbdc5b3288 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:53.641069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be22b5-8b0f-4911-a28a-17fbdc5b3288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:53.699549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a2be22b5-8b0f-4911-a28a-17fbdc5b3288 None None] MFA Rules not processed for user `00aefa0efeb54dcb9e21dc1b8a9a26d8`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:53.700839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a2be22b5-8b0f-4911-a28a-17fbdc5b3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:53.778402 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 889/1750] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:53.779418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05da12-c571-4a41-815d-78b3c958ed90 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:53.877130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 862/1751] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:53.877802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be22b5-8b0f-4911-a28a-17fbdc5b3288 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:53.887761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a2be22b5-8b0f-4911-a28a-17fbdc5b3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:53.924533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be22b5-8b0f-4911-a28a-17fbdc5b3288 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:53.940986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef6615-21e2-4200-a0ff-b4d491202e1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:53.941643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef6615-21e2-4200-a0ff-b4d491202e1c service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:53.941930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef6615-21e2-4200-a0ff-b4d491202e1c service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:53.942307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef6615-21e2-4200-a0ff-b4d491202e1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:53.952458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e5ef6615-21e2-4200-a0ff-b4d491202e1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:53.988155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05da12-c571-4a41-815d-78b3c958ed90 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:54.000700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef6615-21e2-4200-a0ff-b4d491202e1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:54.002441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef6615-21e2-4200-a0ff-b4d491202e1c service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:54.013335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b88a7f-19ca-44a1-ab62-d4b70282fdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:54.018888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b88a7f-19ca-44a1-ab62-d4b70282fdca admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:54.019437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b88a7f-19ca-44a1-ab62-d4b70282fdca admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:54.022701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b88a7f-19ca-44a1-ab62-d4b70282fdca admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:54.025918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b88a7f-19ca-44a1-ab62-d4b70282fdca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:54.036823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b88a7f-19ca-44a1-ab62-d4b70282fdca admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:54.058953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 863/1752] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:54.062362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef6615-21e2-4200-a0ff-b4d491202e1c service placement] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:54.066020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e5ef6615-21e2-4200-a0ff-b4d491202e1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:54.090516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 890/1753] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:54.092406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b88a7f-19ca-44a1-ab62-d4b70282fdca admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:54.180900 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b88a7f-19ca-44a1-ab62-d4b70282fdca admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:54.212175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b20b4-ce32-4fd5-9f97-491e058827f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:54.219463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b20b4-ce32-4fd5-9f97-491e058827f8 service placement] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:54.221555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b20b4-ce32-4fd5-9f97-491e058827f8 service placement] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:54.225645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b20b4-ce32-4fd5-9f97-491e058827f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:54.271732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b20b4-ce32-4fd5-9f97-491e058827f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:54.274059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b20b4-ce32-4fd5-9f97-491e058827f8 service placement] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:54.289828 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 891/1754] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:54.291536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b20b4-ce32-4fd5-9f97-491e058827f8 service placement] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:54.293532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-502b20b4-ce32-4fd5-9f97-491e058827f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:54.327563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b20b4-ce32-4fd5-9f97-491e058827f8 service placement] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:54.348056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:54.361567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:54.363657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:54.367512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] PATH_INFO: `/v3/projects/64f81017c0474297ac64ce306d739071` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:54.371078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f81017c0474297ac64ce306d739071)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:54.377411 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:54.436164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef6615-21e2-4200-a0ff-b4d491202e1c service placement] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:54.456192 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f369-d8e2-42ca-9c30-428a74abddae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:54.457861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f369-d8e2-42ca-9c30-428a74abddae service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:54.457861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f369-d8e2-42ca-9c30-428a74abddae service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:54.463191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f369-d8e2-42ca-9c30-428a74abddae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:54.472201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:54.473769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9a56f369-d8e2-42ca-9c30-428a74abddae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:54.669892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f81017c0474297ac64ce306d739071', 'request_id': 'req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:54.698277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f369-d8e2-42ca-9c30-428a74abddae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:54.700288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f369-d8e2-42ca-9c30-428a74abddae service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:54.704964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f81017c0474297ac64ce306d739071', 'request_id': 'req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:54.730867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 864/1755] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:54.734763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f369-d8e2-42ca-9c30-428a74abddae service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:54.752570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f81017c0474297ac64ce306d739071', 'request_id': 'req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:54.765089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f369-d8e2-42ca-9c30-428a74abddae service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:54.883532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] The token cache is being invalidate because project 64f81017c0474297ac64ce306d739071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:26:54.883818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:54.885107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-17723c2d-cfe0-4a2d-8e2e-bb699e6dd6c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:54.892160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 892/1756] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:26:54 2022] DELETE /identity/v3/projects/64f81017c0474297ac64ce306d739071 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:54.892160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a702825-355d-43e2-962e-6c63b1cfa2a6 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:54.892160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a702825-355d-43e2-962e-6c63b1cfa2a6 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:54.892160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a702825-355d-43e2-962e-6c63b1cfa2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:54.953048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1a702825-355d-43e2-962e-6c63b1cfa2a6 None None] MFA Rules not processed for user `f46cbc7de18c4f9fb6f54e96be53fea0`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:54.958325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1a702825-355d-43e2-962e-6c63b1cfa2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:54.999146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf9809-b346-4f4a-a745-8a89689c7aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:54.999969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf9809-b346-4f4a-a745-8a89689c7aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:55.000204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf9809-b346-4f4a-a745-8a89689c7aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:55.000424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf9809-b346-4f4a-a745-8a89689c7aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:55.009109 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf9809-b346-4f4a-a745-8a89689c7aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:55.012910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf9809-b346-4f4a-a745-8a89689c7aa6 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:55.036579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 865/1757] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:55.038991 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf9809-b346-4f4a-a745-8a89689c7aa6 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:55.044241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9ebf9809-b346-4f4a-a745-8a89689c7aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:55.066610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf9809-b346-4f4a-a745-8a89689c7aa6 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:55.074832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:55.076054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:55.076530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:55.077011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] PATH_INFO: `/v3/users/88769c24fc094b29a29d4a04cfa8c7ab` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:55.079429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] RBAC: Authorizing `identity:delete_user(user_id=88769c24fc094b29a29d4a04cfa8c7ab)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:55.083022 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:55.171274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 893/1758] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:55.171590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:55.171665 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a702825-355d-43e2-962e-6c63b1cfa2a6 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:55.177194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1a702825-355d-43e2-962e-6c63b1cfa2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:55.253813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a702825-355d-43e2-962e-6c63b1cfa2a6 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:55.260393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87f324-e51d-48df-b2c6-49846032241e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:55.261413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87f324-e51d-48df-b2c6-49846032241e admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:55.261748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87f324-e51d-48df-b2c6-49846032241e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:55.262034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87f324-e51d-48df-b2c6-49846032241e admin admin] PATH_INFO: `/v3/projects/24463ed454294efba9cc30c5e6b08298/users/0fe04817e9ff4e8e8d6beedf2bf871a5/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:55.328275 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87f324-e51d-48df-b2c6-49846032241e admin admin] RBAC: Authorizing `identity:create_grant(project_id=24463ed454294efba9cc30c5e6b08298, user_id=0fe04817e9ff4e8e8d6beedf2bf871a5, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:55.331902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87f324-e51d-48df-b2c6-49846032241e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:55.447892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 894/1759] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:55 2022] PUT /identity/v3/projects/24463ed454294efba9cc30c5e6b08298/users/0fe04817e9ff4e8e8d6beedf2bf871a5/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:55.448933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87f324-e51d-48df-b2c6-49846032241e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:55.452323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bd87f324-e51d-48df-b2c6-49846032241e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:55.481162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88769c24fc094b29a29d4a04cfa8c7ab'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:55.482888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88769c24fc094b29a29d4a04cfa8c7ab'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:55.487223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88769c24fc094b29a29d4a04cfa8c7ab'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:55.506980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88769c24fc094b29a29d4a04cfa8c7ab'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:55.537700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f3443bdd-115f-43ac-981b-b78cd0ca123b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88769c24fc094b29a29d4a04cfa8c7ab'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:55.639153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 866/1760] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:55 2022] DELETE /identity/v3/users/88769c24fc094b29a29d4a04cfa8c7ab => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:55.640828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377e5bd-1172-4569-b3f9-1c5857af8bda None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:55.641132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377e5bd-1172-4569-b3f9-1c5857af8bda None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:55.641548 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377e5bd-1172-4569-b3f9-1c5857af8bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:55.730673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c377e5bd-1172-4569-b3f9-1c5857af8bda None None] MFA Rules not processed for user `0fe04817e9ff4e8e8d6beedf2bf871a5`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:55.734169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c377e5bd-1172-4569-b3f9-1c5857af8bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:55.857709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87f324-e51d-48df-b2c6-49846032241e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:55.995060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 867/1761] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:56.004518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377e5bd-1172-4569-b3f9-1c5857af8bda None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:56.010218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c377e5bd-1172-4569-b3f9-1c5857af8bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:56.053777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483d5b9-8de4-48d2-b018-02eb67109e96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:56.054508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483d5b9-8de4-48d2-b018-02eb67109e96 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:56.054788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483d5b9-8de4-48d2-b018-02eb67109e96 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:56.055059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483d5b9-8de4-48d2-b018-02eb67109e96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:56.061821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5483d5b9-8de4-48d2-b018-02eb67109e96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:56.269702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483d5b9-8de4-48d2-b018-02eb67109e96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:56.273627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483d5b9-8de4-48d2-b018-02eb67109e96 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:56.298851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 895/1762] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 850 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:56.301815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377e5bd-1172-4569-b3f9-1c5857af8bda None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:56.324919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69c609-5775-4773-8b10-a0fe2a1f9245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:56.326108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69c609-5775-4773-8b10-a0fe2a1f9245 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:56.326650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69c609-5775-4773-8b10-a0fe2a1f9245 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:56.327458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69c609-5775-4773-8b10-a0fe2a1f9245 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:56.328203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69c609-5775-4773-8b10-a0fe2a1f9245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:56.333274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69c609-5775-4773-8b10-a0fe2a1f9245 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:56.478714 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 868/1763] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:26:56 2022] POST /identity/v3/projects => generated 381 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:56.484229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483d5b9-8de4-48d2-b018-02eb67109e96 service nova] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:56.510554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:56.534044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483d5b9-8de4-48d2-b018-02eb67109e96 service nova] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:56.603064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:56.803296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccfdb5-3158-4a13-a0b9-37fbdb0010d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:56.806405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccfdb5-3158-4a13-a0b9-37fbdb0010d0 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:56.806844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccfdb5-3158-4a13-a0b9-37fbdb0010d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:56.807120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccfdb5-3158-4a13-a0b9-37fbdb0010d0 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:56.807755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccfdb5-3158-4a13-a0b9-37fbdb0010d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:56.813871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccfdb5-3158-4a13-a0b9-37fbdb0010d0 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:57.034657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 896/1764] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:26:56 2022] POST /identity/v3/users => generated 297 bytes in 547 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:26:57.040303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69c609-5775-4773-8b10-a0fe2a1f9245 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:57.113512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69c609-5775-4773-8b10-a0fe2a1f9245 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:57.144992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccfdb5-3158-4a13-a0b9-37fbdb0010d0 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:57.147879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7dccfdb5-3158-4a13-a0b9-37fbdb0010d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:57.186059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e5fe6-2ecf-405a-9952-cbc3b8c4c4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:57.186059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e5fe6-2ecf-405a-9952-cbc3b8c4c4ef admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:57.186059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e5fe6-2ecf-405a-9952-cbc3b8c4c4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:57.198000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e5fe6-2ecf-405a-9952-cbc3b8c4c4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:57.203620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccfdb5-3158-4a13-a0b9-37fbdb0010d0 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:57.207027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e5fe6-2ecf-405a-9952-cbc3b8c4c4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:57.235770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:57.237848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:57.238997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e5fe6-2ecf-405a-9952-cbc3b8c4c4ef admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:57.240894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:57.241629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] PATH_INFO: `/v3/projects/fbcbc15286a249c1b30b197521f61fb2` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:57.245629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbcbc15286a249c1b30b197521f61fb2)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:57.251538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:57.307788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 869/1765] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:26:57.308903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e5fe6-2ecf-405a-9952-cbc3b8c4c4ef admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:57.336356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e5fe6-2ecf-405a-9952-cbc3b8c4c4ef admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:57.373351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:57.376934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e3cdf-0647-46e6-8325-bd2e8abf72be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:57.377625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e3cdf-0647-46e6-8325-bd2e8abf72be admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:57.377907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e3cdf-0647-46e6-8325-bd2e8abf72be admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:57.378175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e3cdf-0647-46e6-8325-bd2e8abf72be admin admin] PATH_INFO: `/v3/projects/e4d476c58e774171aba06d510ebb0be0/users/ce288830f6554e1ebbbe34b6e4233c87/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:57.540237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e3cdf-0647-46e6-8325-bd2e8abf72be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d476c58e774171aba06d510ebb0be0, user_id=ce288830f6554e1ebbbe34b6e4233c87, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:57.550645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e3cdf-0647-46e6-8325-bd2e8abf72be admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:57.660261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 870/1766] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:57 2022] PUT /identity/v3/projects/e4d476c58e774171aba06d510ebb0be0/users/ce288830f6554e1ebbbe34b6e4233c87/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:57.667019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e3cdf-0647-46e6-8325-bd2e8abf72be admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:57.674795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbcbc15286a249c1b30b197521f61fb2', 'request_id': 'req-9b130e31-3e91-40e4-a218-1a175e23c7d3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:57.691909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbcbc15286a249c1b30b197521f61fb2', 'request_id': 'req-9b130e31-3e91-40e4-a218-1a175e23c7d3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:57.710877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbcbc15286a249c1b30b197521f61fb2', 'request_id': 'req-9b130e31-3e91-40e4-a218-1a175e23c7d3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:58.115753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] The token cache is being invalidate because project fbcbc15286a249c1b30b197521f61fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:26:58.115753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:58.115753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:58.115753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 897/1767] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:26:57 2022] DELETE /identity/v3/projects/fbcbc15286a249c1b30b197521f61fb2 => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:58.117431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e3cdf-0647-46e6-8325-bd2e8abf72be admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:58.117431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ccfcd-1770-42fb-99b0-1730d7b3746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:58.117431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ccfcd-1770-42fb-99b0-1730d7b3746d admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:58.117431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ccfcd-1770-42fb-99b0-1730d7b3746d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:58.118747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ccfcd-1770-42fb-99b0-1730d7b3746d admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:58.118747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ccfcd-1770-42fb-99b0-1730d7b3746d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:58.118747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ccfcd-1770-42fb-99b0-1730d7b3746d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:58.118747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 871/1768] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:26:58.118747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ccfcd-1770-42fb-99b0-1730d7b3746d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:58.118747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c24ccfcd-1770-42fb-99b0-1730d7b3746d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:58.119646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:58.119646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:58.127675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ccfcd-1770-42fb-99b0-1730d7b3746d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:58.153745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d154b-e499-4be0-8d3e-98361202ae69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:58.155096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d154b-e499-4be0-8d3e-98361202ae69 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:58.155604 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d154b-e499-4be0-8d3e-98361202ae69 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:58.156048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d154b-e499-4be0-8d3e-98361202ae69 admin admin] PATH_INFO: `/v3/projects/e4d476c58e774171aba06d510ebb0be0/users/ce288830f6554e1ebbbe34b6e4233c87/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:58.159249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d154b-e499-4be0-8d3e-98361202ae69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d476c58e774171aba06d510ebb0be0, user_id=ce288830f6554e1ebbbe34b6e4233c87, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:58.164231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d154b-e499-4be0-8d3e-98361202ae69 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:58.180226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 872/1769] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:26:58 2022] PUT /identity/v3/projects/e4d476c58e774171aba06d510ebb0be0/users/ce288830f6554e1ebbbe34b6e4233c87/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:58.211622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82942b1a-db6e-487a-814c-1637ef399688 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:58.212248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82942b1a-db6e-487a-814c-1637ef399688 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:58.212844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82942b1a-db6e-487a-814c-1637ef399688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:58.278988 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b130e31-3e91-40e4-a218-1a175e23c7d3 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:58.316802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-82942b1a-db6e-487a-814c-1637ef399688 None None] MFA Rules not processed for user `ce288830f6554e1ebbbe34b6e4233c87`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:58.323006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-82942b1a-db6e-487a-814c-1637ef399688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:58.438359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:58.438359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:58.438359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:58.438359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] PATH_INFO: `/v3/users/def8351bd62947ef9dfd10374f54dca4` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:58.439419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=def8351bd62947ef9dfd10374f54dca4)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:58.441797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:58.670175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 873/1770] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:58.671903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82942b1a-db6e-487a-814c-1637ef399688 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:58.674851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-82942b1a-db6e-487a-814c-1637ef399688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:58.727088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82942b1a-db6e-487a-814c-1637ef399688 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:58.736862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:58.738529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:58.738963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:58.739590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] PATH_INFO: `/v3/users/00aefa0efeb54dcb9e21dc1b8a9a26d8` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:58.743585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=00aefa0efeb54dcb9e21dc1b8a9a26d8)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:58.744676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def8351bd62947ef9dfd10374f54dca4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:58.747125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def8351bd62947ef9dfd10374f54dca4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:58.749886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def8351bd62947ef9dfd10374f54dca4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:58.750609 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:58.859379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def8351bd62947ef9dfd10374f54dca4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:58.873032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def8351bd62947ef9dfd10374f54dca4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:58.915642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 898/1771] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:58 2022] DELETE /identity/v3/users/def8351bd62947ef9dfd10374f54dca4 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:58.939926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:58.946546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:59.125465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7363085-7da8-4a83-b9ab-a9d391622a20 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:59.133081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a14c6-8c72-43f6-853d-ca656a68bd05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:59.134728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a14c6-8c72-43f6-853d-ca656a68bd05 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:59.135576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a14c6-8c72-43f6-853d-ca656a68bd05 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:59.136095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a14c6-8c72-43f6-853d-ca656a68bd05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:59.139518 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-360a14c6-8c72-43f6-853d-ca656a68bd05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:59.196984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00aefa0efeb54dcb9e21dc1b8a9a26d8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:59.198177 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00aefa0efeb54dcb9e21dc1b8a9a26d8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:59.198906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00aefa0efeb54dcb9e21dc1b8a9a26d8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:59.208107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00aefa0efeb54dcb9e21dc1b8a9a26d8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:59.222653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00aefa0efeb54dcb9e21dc1b8a9a26d8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:26:59.266025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:59.278311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a14c6-8c72-43f6-853d-ca656a68bd05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:59.280792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a14c6-8c72-43f6-853d-ca656a68bd05 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:59.301599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 899/1772] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:26:59.420739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 874/1773] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:26:58 2022] DELETE /identity/v3/users/00aefa0efeb54dcb9e21dc1b8a9a26d8 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:26:59.427245 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774c10c-e25e-4033-9719-341112bb7402 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:59.428188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774c10c-e25e-4033-9719-341112bb7402 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:59.428563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774c10c-e25e-4033-9719-341112bb7402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:59.464720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1774c10c-e25e-4033-9719-341112bb7402 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:26:59.466274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1774c10c-e25e-4033-9719-341112bb7402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:59.586502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 900/1774] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:26:59.596011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:26:59.598590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:26:59.701154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:26:59.809295 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e080e7f-4e28-4845-8511-786cdefa70dd admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:26:59.815476 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a98d9-c83f-4751-bd64-9882544ab066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:26:59.816139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a98d9-c83f-4751-bd64-9882544ab066 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:26:59.816444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a98d9-c83f-4751-bd64-9882544ab066 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:26:59.816712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a98d9-c83f-4751-bd64-9882544ab066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:26:59.829418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a98d9-c83f-4751-bd64-9882544ab066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:26:59.831105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a98d9-c83f-4751-bd64-9882544ab066 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:26:59.847472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 875/1775] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:26:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:00.242321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774c10c-e25e-4033-9719-341112bb7402 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:00.243851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1774c10c-e25e-4033-9719-341112bb7402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:00.258071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774c10c-e25e-4033-9719-341112bb7402 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:00.264718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:00.265568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:00.265949 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:00.266463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] PATH_INFO: `/v3/projects/9df2e90531504e20bd9c1461f8d3f126` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:00.269689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df2e90531504e20bd9c1461f8d3f126)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:00.273109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:00.487075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df2e90531504e20bd9c1461f8d3f126', 'request_id': 'req-0134c2db-374d-44a9-b784-5da93e2d1e9a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:00.492954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a98d9-c83f-4751-bd64-9882544ab066 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:00.496031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-840a98d9-c83f-4751-bd64-9882544ab066 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:00.571811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df2e90531504e20bd9c1461f8d3f126', 'request_id': 'req-0134c2db-374d-44a9-b784-5da93e2d1e9a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:00.592294 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a98d9-c83f-4751-bd64-9882544ab066 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:00.625039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df2e90531504e20bd9c1461f8d3f126', 'request_id': 'req-0134c2db-374d-44a9-b784-5da93e2d1e9a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:00.681165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:00.774251 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:00.776749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:00.777571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:00.778329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] PATH_INFO: `/v3/users/0fe04817e9ff4e8e8d6beedf2bf871a5` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:00.781593 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe04817e9ff4e8e8d6beedf2bf871a5)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:00.800205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:00.881272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:00.899507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] The token cache is being invalidate because project 9df2e90531504e20bd9c1461f8d3f126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:00.899507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:00.901259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:00.904907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 901/1776] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:00 2022] DELETE /identity/v3/projects/9df2e90531504e20bd9c1461f8d3f126 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:00.907175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:00.910609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:00.947890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134c2db-374d-44a9-b784-5da93e2d1e9a admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:00.971926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:00.988573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:00.988573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:00.988573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] PATH_INFO: `/v3/projects/0179c56cebe24f9fbe5d907c375f6ab0` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:00.988573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0179c56cebe24f9fbe5d907c375f6ab0)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:01.016260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:01.215354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe04817e9ff4e8e8d6beedf2bf871a5'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:01.216526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe04817e9ff4e8e8d6beedf2bf871a5'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:01.218122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe04817e9ff4e8e8d6beedf2bf871a5'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:01.264483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe04817e9ff4e8e8d6beedf2bf871a5'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:01.270895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0179c56cebe24f9fbe5d907c375f6ab0', 'request_id': 'req-07a15920-dbfb-4231-9382-27e4eb76bebc'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:01.283401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0179c56cebe24f9fbe5d907c375f6ab0', 'request_id': 'req-07a15920-dbfb-4231-9382-27e4eb76bebc'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:01.312473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0179c56cebe24f9fbe5d907c375f6ab0', 'request_id': 'req-07a15920-dbfb-4231-9382-27e4eb76bebc'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:01.342755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-0242b994-1f68-4dfd-bb82-9649eab07ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe04817e9ff4e8e8d6beedf2bf871a5'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:01.464091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] The token cache is being invalidate because project 0179c56cebe24f9fbe5d907c375f6ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:01.464091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:01.464091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-07a15920-dbfb-4231-9382-27e4eb76bebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:01.470986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 902/1777] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:00 2022] DELETE /identity/v3/projects/0179c56cebe24f9fbe5d907c375f6ab0 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:01.478923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9697-94c6-453a-bf65-ed81221041f1 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:01.482902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9697-94c6-453a-bf65-ed81221041f1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:01.489005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9697-94c6-453a-bf65-ed81221041f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:01.565091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-436d9697-94c6-453a-bf65-ed81221041f1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:01.567770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-436d9697-94c6-453a-bf65-ed81221041f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:01.606518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 876/1778] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:00 2022] DELETE /identity/v3/users/0fe04817e9ff4e8e8d6beedf2bf871a5 => generated 0 bytes in 1116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:01.619103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5472f-0e3c-49aa-9fb3-4dfcf5b9ad46 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:01.631691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5472f-0e3c-49aa-9fb3-4dfcf5b9ad46 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:01.632940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5472f-0e3c-49aa-9fb3-4dfcf5b9ad46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:01.727239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-02d5472f-0e3c-49aa-9fb3-4dfcf5b9ad46 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:01.729361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-02d5472f-0e3c-49aa-9fb3-4dfcf5b9ad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:01.814587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 903/1779] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:01.814587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acd099-109a-4d0d-8fab-5754d0c13f90 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:01.814587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acd099-109a-4d0d-8fab-5754d0c13f90 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:01.814587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acd099-109a-4d0d-8fab-5754d0c13f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:01.861450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-34acd099-109a-4d0d-8fab-5754d0c13f90 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:01.863018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-34acd099-109a-4d0d-8fab-5754d0c13f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:01.960710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 904/1780] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:01.960710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025dfe4-6d04-4644-99cb-717fb7a3877d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:01.960710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025dfe4-6d04-4644-99cb-717fb7a3877d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:01.960710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025dfe4-6d04-4644-99cb-717fb7a3877d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:01.979703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 877/1781] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:01.986204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5472f-0e3c-49aa-9fb3-4dfcf5b9ad46 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:01.989434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-02d5472f-0e3c-49aa-9fb3-4dfcf5b9ad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:02.006281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-a025dfe4-6d04-4644-99cb-717fb7a3877d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:02.007944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a025dfe4-6d04-4644-99cb-717fb7a3877d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:02.028455 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 905/1782] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:02.031214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4480c8-3430-420c-b3eb-1505acdc0d6f None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.031501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4480c8-3430-420c-b3eb-1505acdc0d6f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.031779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4480c8-3430-420c-b3eb-1505acdc0d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.071942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6b4480c8-3430-420c-b3eb-1505acdc0d6f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:02.074318 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6b4480c8-3430-420c-b3eb-1505acdc0d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:02.087740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 906/1783] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:02.088620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4480c8-3430-420c-b3eb-1505acdc0d6f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:02.104661 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4480c8-3430-420c-b3eb-1505acdc0d6f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.121667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475ee69-45b5-44e2-b553-6de6b9d02666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.123105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475ee69-45b5-44e2-b553-6de6b9d02666 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.124188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475ee69-45b5-44e2-b553-6de6b9d02666 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.124848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475ee69-45b5-44e2-b553-6de6b9d02666 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.137508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475ee69-45b5-44e2-b553-6de6b9d02666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:27:02.141418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475ee69-45b5-44e2-b553-6de6b9d02666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:02.147216 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5472f-0e3c-49aa-9fb3-4dfcf5b9ad46 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.149659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475ee69-45b5-44e2-b553-6de6b9d02666 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:02.157218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a39ca3-94be-4728-9a37-f5d9f06ad7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.158246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a39ca3-94be-4728-9a37-f5d9f06ad7fa service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.159098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a39ca3-94be-4728-9a37-f5d9f06ad7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.166271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a39ca3-94be-4728-9a37-f5d9f06ad7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.173919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a39ca3-94be-4728-9a37-f5d9f06ad7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:02.176706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a39ca3-94be-4728-9a37-f5d9f06ad7fa service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:02.195274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 878/1784] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:02.200524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 907/1785] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:27:02.201029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a39ca3-94be-4728-9a37-f5d9f06ad7fa service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:02.202428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475ee69-45b5-44e2-b553-6de6b9d02666 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:02.216054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a39ca3-94be-4728-9a37-f5d9f06ad7fa service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.222446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc93b3-cd92-420d-a1c4-0541f695d7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.223491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc93b3-cd92-420d-a1c4-0541f695d7ae admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.223969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc93b3-cd92-420d-a1c4-0541f695d7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.224495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc93b3-cd92-420d-a1c4-0541f695d7ae admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.225629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc93b3-cd92-420d-a1c4-0541f695d7ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:27:02.226166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc93b3-cd92-420d-a1c4-0541f695d7ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:02.227860 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475ee69-45b5-44e2-b553-6de6b9d02666 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.230954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc93b3-cd92-420d-a1c4-0541f695d7ae admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:02.243200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc294ec3-6073-41cf-90da-b37c0a89d297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.243848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc294ec3-6073-41cf-90da-b37c0a89d297 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.244168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc294ec3-6073-41cf-90da-b37c0a89d297 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.245995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc294ec3-6073-41cf-90da-b37c0a89d297 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.246711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc294ec3-6073-41cf-90da-b37c0a89d297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:02.254707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc294ec3-6073-41cf-90da-b37c0a89d297 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:02.260863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:02.276392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 879/1786] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:27:02.281830 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc93b3-cd92-420d-a1c4-0541f695d7ae admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:02.343870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc93b3-cd92-420d-a1c4-0541f695d7ae admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.365774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b79d93-785c-4b84-9935-05bdefe523c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.367180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b79d93-785c-4b84-9935-05bdefe523c3 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.367652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b79d93-785c-4b84-9935-05bdefe523c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.368074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b79d93-785c-4b84-9935-05bdefe523c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.371004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b79d93-785c-4b84-9935-05bdefe523c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:02.373799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b79d93-785c-4b84-9935-05bdefe523c3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:02.375634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:02.461072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 908/1787] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:02 2022] POST /identity/v3/projects => generated 367 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:02.462824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc294ec3-6073-41cf-90da-b37c0a89d297 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:02.478467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 880/1788] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:02 2022] POST /identity/v3/projects => generated 405 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:02.497701 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b79d93-785c-4b84-9935-05bdefe523c3 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:02.676740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc294ec3-6073-41cf-90da-b37c0a89d297 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.677832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b79d93-785c-4b84-9935-05bdefe523c3 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.679059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6d92a-1d24-4b70-b947-34fb86e58c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.680255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6d92a-1d24-4b70-b947-34fb86e58c52 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.680581 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6d92a-1d24-4b70-b947-34fb86e58c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.681004 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6d92a-1d24-4b70-b947-34fb86e58c52 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.681989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6d92a-1d24-4b70-b947-34fb86e58c52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:02.683696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eab2e-a591-4471-9116-06252dc2a4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.684330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eab2e-a591-4471-9116-06252dc2a4bf admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.684603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eab2e-a591-4471-9116-06252dc2a4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.684905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eab2e-a591-4471-9116-06252dc2a4bf admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.685257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6d92a-1d24-4b70-b947-34fb86e58c52 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:02.685613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eab2e-a591-4471-9116-06252dc2a4bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:02.688652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eab2e-a591-4471-9116-06252dc2a4bf admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:02.859808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 881/1789] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:02 2022] POST /identity/v3/users => generated 303 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:02.862536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 909/1790] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:02 2022] POST /identity/v3/users => generated 284 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:02.872289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eab2e-a591-4471-9116-06252dc2a4bf admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:02.872764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6d92a-1d24-4b70-b947-34fb86e58c52 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:02.910648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eab2e-a591-4471-9116-06252dc2a4bf admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.915325 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6d92a-1d24-4b70-b947-34fb86e58c52 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.922961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd8019-3051-4042-b4ec-c964546e7fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.924325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd8019-3051-4042-b4ec-c964546e7fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.925048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd8019-3051-4042-b4ec-c964546e7fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.926231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd8019-3051-4042-b4ec-c964546e7fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.927350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd2ac9-7a14-43ba-9b7e-ea6a2367d22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.946633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd8019-3051-4042-b4ec-c964546e7fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:02.946739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd2ac9-7a14-43ba-9b7e-ea6a2367d22f admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.946739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd2ac9-7a14-43ba-9b7e-ea6a2367d22f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.946739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd2ac9-7a14-43ba-9b7e-ea6a2367d22f admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.946739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd2ac9-7a14-43ba-9b7e-ea6a2367d22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:02.946739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd2ac9-7a14-43ba-9b7e-ea6a2367d22f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:02.947081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd8019-3051-4042-b4ec-c964546e7fa4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:02.961894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 882/1791] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:02.964993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 910/1792] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:02.970943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd8019-3051-4042-b4ec-c964546e7fa4 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:02.972107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd2ac9-7a14-43ba-9b7e-ea6a2367d22f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:02.985119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd2ac9-7a14-43ba-9b7e-ea6a2367d22f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.985661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd8019-3051-4042-b4ec-c964546e7fa4 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:02.992620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fd1df-89c2-47cb-888d-6961b5ad6d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.993396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fd1df-89c2-47cb-888d-6961b5ad6d2e admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.993679 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fd1df-89c2-47cb-888d-6961b5ad6d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.993953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fd1df-89c2-47cb-888d-6961b5ad6d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.994200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bf6d1-d2d8-44c8-a053-5f4eb83167d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:02.994873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842fd1df-89c2-47cb-888d-6961b5ad6d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:02.995445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bf6d1-d2d8-44c8-a053-5f4eb83167d6 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:02.996142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bf6d1-d2d8-44c8-a053-5f4eb83167d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:02.996785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bf6d1-d2d8-44c8-a053-5f4eb83167d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:02.997087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842fd1df-89c2-47cb-888d-6961b5ad6d2e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:02.998772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bf6d1-d2d8-44c8-a053-5f4eb83167d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:03.001642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bf6d1-d2d8-44c8-a053-5f4eb83167d6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:03.021224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 911/1793] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:03.023129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bf6d1-d2d8-44c8-a053-5f4eb83167d6 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:03.030628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 883/1794] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:03.035058 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fd1df-89c2-47cb-888d-6961b5ad6d2e admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:03.041297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bf6d1-d2d8-44c8-a053-5f4eb83167d6 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:03.045150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fd1df-89c2-47cb-888d-6961b5ad6d2e admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:03.047630 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d04112-81e6-45a4-b6db-e855fa10830e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:03.048338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d04112-81e6-45a4-b6db-e855fa10830e admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:03.048649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d04112-81e6-45a4-b6db-e855fa10830e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:03.048893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d04112-81e6-45a4-b6db-e855fa10830e admin admin] PATH_INFO: `/v3/projects/ddfb94fcd2924c39b0268fcb0a89cd8b/users/d0481f30d54544de82589fab75ad8593/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:03.055646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ab82f-232d-449d-9929-09765ee0df3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:03.055646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ab82f-232d-449d-9929-09765ee0df3b admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:03.056062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ab82f-232d-449d-9929-09765ee0df3b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:03.056062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ab82f-232d-449d-9929-09765ee0df3b admin admin] PATH_INFO: `/v3/projects/8c1c4f3e5c114af7b6fcbbebd9ee754c/users/0b2a968526f443a2be990b48bf18c503/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:03.086269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ab82f-232d-449d-9929-09765ee0df3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1c4f3e5c114af7b6fcbbebd9ee754c, user_id=0b2a968526f443a2be990b48bf18c503, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:03.088883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d04112-81e6-45a4-b6db-e855fa10830e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddfb94fcd2924c39b0268fcb0a89cd8b, user_id=d0481f30d54544de82589fab75ad8593, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:03.089153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ab82f-232d-449d-9929-09765ee0df3b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:03.097112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d04112-81e6-45a4-b6db-e855fa10830e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:03.219013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 884/1795] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:03 2022] PUT /identity/v3/projects/8c1c4f3e5c114af7b6fcbbebd9ee754c/users/0b2a968526f443a2be990b48bf18c503/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:03.220073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ab82f-232d-449d-9929-09765ee0df3b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:03.222040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3d5ab82f-232d-449d-9929-09765ee0df3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:03.251158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 912/1796] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:03 2022] PUT /identity/v3/projects/ddfb94fcd2924c39b0268fcb0a89cd8b/users/d0481f30d54544de82589fab75ad8593/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:03.253163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eaf03-2104-44b0-ba03-dccb7d826883 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:03.253577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eaf03-2104-44b0-ba03-dccb7d826883 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:03.253968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eaf03-2104-44b0-ba03-dccb7d826883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:03.277841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-443eaf03-2104-44b0-ba03-dccb7d826883 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:03.277841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-443eaf03-2104-44b0-ba03-dccb7d826883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:03.278359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:03.293388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:03.319011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ab82f-232d-449d-9929-09765ee0df3b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:03.361912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 913/1797] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:03.363659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479d7cd-2d35-4441-afbe-8e15a7cdfa04 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:03.363951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479d7cd-2d35-4441-afbe-8e15a7cdfa04 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:03.364230 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479d7cd-2d35-4441-afbe-8e15a7cdfa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:03.402544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:03.403374 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:03.403658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:03.409144 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] PATH_INFO: `/v3/projects/24463ed454294efba9cc30c5e6b08298` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:03.409144 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=24463ed454294efba9cc30c5e6b08298)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:03.409144 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:03.438677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-7479d7cd-2d35-4441-afbe-8e15a7cdfa04 None None] MFA Rules not processed for user `d0481f30d54544de82589fab75ad8593`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:03.440281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7479d7cd-2d35-4441-afbe-8e15a7cdfa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:03.612971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24463ed454294efba9cc30c5e6b08298', 'request_id': 'req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:03.631510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24463ed454294efba9cc30c5e6b08298', 'request_id': 'req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:03.670027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 914/1798] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:03.681952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479d7cd-2d35-4441-afbe-8e15a7cdfa04 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:03.752932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24463ed454294efba9cc30c5e6b08298', 'request_id': 'req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:03.778132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] The token cache is being invalidate because project 24463ed454294efba9cc30c5e6b08298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:03.778633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:03.780756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:03.793938 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 885/1799] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:03 2022] DELETE /identity/v3/projects/24463ed454294efba9cc30c5e6b08298 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:03.797696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:03.803256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:03.923557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479d7cd-2d35-4441-afbe-8e15a7cdfa04 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:03.993930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bda43-9c4c-4ffa-be49-40de3ad09c5f admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:04.011489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb5c0f-dd10-4366-a4ed-1df148a9dc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:04.012155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb5c0f-dd10-4366-a4ed-1df148a9dc27 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:04.012435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb5c0f-dd10-4366-a4ed-1df148a9dc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:04.012716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb5c0f-dd10-4366-a4ed-1df148a9dc27 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:04.014126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb5c0f-dd10-4366-a4ed-1df148a9dc27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:04.021572 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb5c0f-dd10-4366-a4ed-1df148a9dc27 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:04.163104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39bf56-084b-482f-808b-ccd219672205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:04.164690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39bf56-084b-482f-808b-ccd219672205 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:04.165465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39bf56-084b-482f-808b-ccd219672205 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:04.168434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39bf56-084b-482f-808b-ccd219672205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:04.176075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8a39bf56-084b-482f-808b-ccd219672205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:04.193791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39bf56-084b-482f-808b-ccd219672205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:04.195788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39bf56-084b-482f-808b-ccd219672205 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:04.208176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 915/1800] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:04.210782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39bf56-084b-482f-808b-ccd219672205 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:04.212929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8a39bf56-084b-482f-808b-ccd219672205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:04.238552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39bf56-084b-482f-808b-ccd219672205 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:04.238876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:04.246543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 886/1801] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:03 2022] POST /identity/v3/projects => generated 367 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:04.249136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb5c0f-dd10-4366-a4ed-1df148a9dc27 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:04.261130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:04.262241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:04.262783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:04.263262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] PATH_INFO: `/v3/users/ce288830f6554e1ebbbe34b6e4233c87` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:04.268450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce288830f6554e1ebbbe34b6e4233c87)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:04.278428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:04.419898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb5c0f-dd10-4366-a4ed-1df148a9dc27 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:04.436631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a6096-4380-4960-a797-dadf143ca18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:04.439824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a6096-4380-4960-a797-dadf143ca18b admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:04.440463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a6096-4380-4960-a797-dadf143ca18b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:04.440919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a6096-4380-4960-a797-dadf143ca18b admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:04.442764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a6096-4380-4960-a797-dadf143ca18b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:04.448952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a6096-4380-4960-a797-dadf143ca18b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:04.552789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 887/1802] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:04 2022] POST /identity/v3/users => generated 290 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:04.555740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a6096-4380-4960-a797-dadf143ca18b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:04.573043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a6096-4380-4960-a797-dadf143ca18b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:04.589554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55a994-9435-43ce-a11e-7ef79c86ae63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:04.593809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55a994-9435-43ce-a11e-7ef79c86ae63 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:04.593809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55a994-9435-43ce-a11e-7ef79c86ae63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:04.593809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55a994-9435-43ce-a11e-7ef79c86ae63 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:04.593809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55a994-9435-43ce-a11e-7ef79c86ae63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:04.593809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55a994-9435-43ce-a11e-7ef79c86ae63 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:04.630208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce288830f6554e1ebbbe34b6e4233c87'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:04.635264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce288830f6554e1ebbbe34b6e4233c87'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:04.643665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 888/1803] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:27:04.643879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce288830f6554e1ebbbe34b6e4233c87'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:04.644048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb15d2d-75d8-42ff-928c-c90ee011a16f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:04.644048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb15d2d-75d8-42ff-928c-c90ee011a16f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:04.644048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb15d2d-75d8-42ff-928c-c90ee011a16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:04.672799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:04.718323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce288830f6554e1ebbbe34b6e4233c87'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:04.734964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce288830f6554e1ebbbe34b6e4233c87'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:04.811594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 916/1804] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:04 2022] DELETE /identity/v3/users/ce288830f6554e1ebbbe34b6e4233c87 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:04.811594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:04.816382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:04.853859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-bfb15d2d-75d8-42ff-928c-c90ee011a16f None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:04.856907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bfb15d2d-75d8-42ff-928c-c90ee011a16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:04.971270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afe654-fa62-41e2-b659-5f4b4fd59e28 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:04.981418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fd666-4be3-41ce-80b9-1b38d7476fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:04.982166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fd666-4be3-41ce-80b9-1b38d7476fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:04.982557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fd666-4be3-41ce-80b9-1b38d7476fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:04.983451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fd666-4be3-41ce-80b9-1b38d7476fa5 admin admin] PATH_INFO: `/v3/projects/cf0b282744614428aa2e34957bb941e4/users/aaae4c51606e454f9836f3f84411fcaa/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:04.992307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 889/1805] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:27:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:05.008114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fd666-4be3-41ce-80b9-1b38d7476fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0b282744614428aa2e34957bb941e4, user_id=aaae4c51606e454f9836f3f84411fcaa, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:05.011794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fd666-4be3-41ce-80b9-1b38d7476fa5 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:05.260635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 917/1806] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:04 2022] PUT /identity/v3/projects/cf0b282744614428aa2e34957bb941e4/users/aaae4c51606e454f9836f3f84411fcaa/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:05.268241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb15d2d-75d8-42ff-928c-c90ee011a16f None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:05.296956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb15d2d-75d8-42ff-928c-c90ee011a16f None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:05.417609 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabcbd12-2c41-4454-a946-55d296a38e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:05.418692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcbd12-2c41-4454-a946-55d296a38e29 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:05.419151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcbd12-2c41-4454-a946-55d296a38e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:05.419605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcbd12-2c41-4454-a946-55d296a38e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:05.420740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabcbd12-2c41-4454-a946-55d296a38e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:05.423784 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabcbd12-2c41-4454-a946-55d296a38e29 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:05.443651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 890/1807] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:05.448398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fd666-4be3-41ce-80b9-1b38d7476fa5 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:05.461966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fd666-4be3-41ce-80b9-1b38d7476fa5 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:05.468373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717dde6-9c9e-42c5-8631-1c2c7e7f2ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:05.469213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717dde6-9c9e-42c5-8631-1c2c7e7f2ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:05.469670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717dde6-9c9e-42c5-8631-1c2c7e7f2ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:05.470029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717dde6-9c9e-42c5-8631-1c2c7e7f2ccd admin admin] PATH_INFO: `/v3/projects/cf0b282744614428aa2e34957bb941e4/users/aaae4c51606e454f9836f3f84411fcaa/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:05.472512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717dde6-9c9e-42c5-8631-1c2c7e7f2ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0b282744614428aa2e34957bb941e4, user_id=aaae4c51606e454f9836f3f84411fcaa, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:05.477849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717dde6-9c9e-42c5-8631-1c2c7e7f2ccd admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:05.502556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 918/1808] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:05 2022] PUT /identity/v3/projects/cf0b282744614428aa2e34957bb941e4/users/aaae4c51606e454f9836f3f84411fcaa/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:05.520044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52affefc-c535-45a6-b1b3-ab023e64540b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:05.520327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52affefc-c535-45a6-b1b3-ab023e64540b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:05.520601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52affefc-c535-45a6-b1b3-ab023e64540b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:05.570869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-52affefc-c535-45a6-b1b3-ab023e64540b None None] MFA Rules not processed for user `aaae4c51606e454f9836f3f84411fcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:05.573653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-52affefc-c535-45a6-b1b3-ab023e64540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:05.697344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 891/1809] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:05.715338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717dde6-9c9e-42c5-8631-1c2c7e7f2ccd admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:05.718858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7717dde6-9c9e-42c5-8631-1c2c7e7f2ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:05.819763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717dde6-9c9e-42c5-8631-1c2c7e7f2ccd admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:05.822210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52affefc-c535-45a6-b1b3-ab023e64540b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:05.824029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-52affefc-c535-45a6-b1b3-ab023e64540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:05.830549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94fb0c-357a-4acf-9b4c-4a3f1f7f3432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:05.830549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94fb0c-357a-4acf-9b4c-4a3f1f7f3432 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:05.830549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94fb0c-357a-4acf-9b4c-4a3f1f7f3432 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:05.830549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94fb0c-357a-4acf-9b4c-4a3f1f7f3432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:05.851393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94fb0c-357a-4acf-9b4c-4a3f1f7f3432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:05.854043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94fb0c-357a-4acf-9b4c-4a3f1f7f3432 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:05.895371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 919/1810] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:27:06.002185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52affefc-c535-45a6-b1b3-ab023e64540b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:06.019698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54129c77-4509-408f-a403-bf499941f374 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:06.020809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54129c77-4509-408f-a403-bf499941f374 service swift] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:06.021514 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54129c77-4509-408f-a403-bf499941f374 service swift] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:06.022375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54129c77-4509-408f-a403-bf499941f374 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:06.167806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54129c77-4509-408f-a403-bf499941f374 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:06.170452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54129c77-4509-408f-a403-bf499941f374 service swift] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:06.183730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 892/1811] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:27:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 17:27:06.378533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94fb0c-357a-4acf-9b4c-4a3f1f7f3432 service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:06.391431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:06.397584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94fb0c-357a-4acf-9b4c-4a3f1f7f3432 service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:06.402260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2731e39-0f02-4e9f-8ee0-722a1bbabcc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:06.403554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2731e39-0f02-4e9f-8ee0-722a1bbabcc2 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:06.404236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2731e39-0f02-4e9f-8ee0-722a1bbabcc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:06.404776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2731e39-0f02-4e9f-8ee0-722a1bbabcc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:06.407604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e2731e39-0f02-4e9f-8ee0-722a1bbabcc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:06.573193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2731e39-0f02-4e9f-8ee0-722a1bbabcc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:06.575033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2731e39-0f02-4e9f-8ee0-722a1bbabcc2 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:06.603039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 920/1812] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:27:06.976998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:07.347100 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54129c77-4509-408f-a403-bf499941f374 service swift] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:07.352494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-54129c77-4509-408f-a403-bf499941f374 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:07.392279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:07.733877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54129c77-4509-408f-a403-bf499941f374 service swift] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:07.748460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:07.749203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:07.749999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:07.750275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] PATH_INFO: `/v3/projects/e4d476c58e774171aba06d510ebb0be0` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:07.751722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d476c58e774171aba06d510ebb0be0)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:07.753701 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:07.932155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d476c58e774171aba06d510ebb0be0', 'request_id': 'req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:07.970558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d476c58e774171aba06d510ebb0be0', 'request_id': 'req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:08.089101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d476c58e774171aba06d510ebb0be0', 'request_id': 'req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:08.126281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] The token cache is being invalidate because project e4d476c58e774171aba06d510ebb0be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:08.126745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:08.128173 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-88ee5dda-ad1d-4d8f-bfc6-378016c402f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:08.134567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 893/1813] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:07 2022] DELETE /identity/v3/projects/e4d476c58e774171aba06d510ebb0be0 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:08.140141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23ce58-80d5-4c4e-9891-efc4de729149 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:08.141013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23ce58-80d5-4c4e-9891-efc4de729149 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:08.141013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23ce58-80d5-4c4e-9891-efc4de729149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:08.258322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-cb23ce58-80d5-4c4e-9891-efc4de729149 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:08.259284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-cb23ce58-80d5-4c4e-9891-efc4de729149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:08.452817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:08.521905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 921/1814] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:08.535944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767dfdf-db47-454c-9da7-ee59b18f629f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:08.538029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767dfdf-db47-454c-9da7-ee59b18f629f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:08.538812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767dfdf-db47-454c-9da7-ee59b18f629f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:08.589241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1767dfdf-db47-454c-9da7-ee59b18f629f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:08.594349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1767dfdf-db47-454c-9da7-ee59b18f629f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:08.617031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 894/1815] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:08.624280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23ce58-80d5-4c4e-9891-efc4de729149 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:08.659408 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23ce58-80d5-4c4e-9891-efc4de729149 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:08.667751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bfbe7-4daf-4bbe-a02c-8ea94030ba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:08.668693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bfbe7-4daf-4bbe-a02c-8ea94030ba00 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:08.669157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bfbe7-4daf-4bbe-a02c-8ea94030ba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:08.669834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bfbe7-4daf-4bbe-a02c-8ea94030ba00 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:08.670989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bfbe7-4daf-4bbe-a02c-8ea94030ba00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:27:08.671707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bfbe7-4daf-4bbe-a02c-8ea94030ba00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:08.674788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bfbe7-4daf-4bbe-a02c-8ea94030ba00 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:08.703066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 922/1816] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:27:08.704662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767dfdf-db47-454c-9da7-ee59b18f629f None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:08.717608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767dfdf-db47-454c-9da7-ee59b18f629f None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:08.723202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82c72a-bb1e-4221-86db-acff77d3fdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:08.723932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82c72a-bb1e-4221-86db-acff77d3fdc2 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:08.724900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82c72a-bb1e-4221-86db-acff77d3fdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:08.725225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82c72a-bb1e-4221-86db-acff77d3fdc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:08.726233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82c72a-bb1e-4221-86db-acff77d3fdc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:08.728429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82c72a-bb1e-4221-86db-acff77d3fdc2 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:08.916577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053832a-2d0d-4651-add2-7137a78a1793 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:08.916999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053832a-2d0d-4651-add2-7137a78a1793 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:08.917446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053832a-2d0d-4651-add2-7137a78a1793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:08.928057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 895/1817] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:08 2022] POST /identity/v3/projects => generated 371 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:08.938914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82c72a-bb1e-4221-86db-acff77d3fdc2 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:09.012487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-3053832a-2d0d-4651-add2-7137a78a1793 None None] MFA Rules not processed for user `0b2a968526f443a2be990b48bf18c503`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:09.015964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3053832a-2d0d-4651-add2-7137a78a1793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:09.198576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 923/1818] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:08 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:09.207349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053832a-2d0d-4651-add2-7137a78a1793 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:09.211903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3053832a-2d0d-4651-add2-7137a78a1793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:09.231500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:09.239155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82c72a-bb1e-4221-86db-acff77d3fdc2 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:09.247166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762c27b-26db-4954-8c9f-20eecd742c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:09.247828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762c27b-26db-4954-8c9f-20eecd742c72 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:09.249059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762c27b-26db-4954-8c9f-20eecd742c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:09.249342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762c27b-26db-4954-8c9f-20eecd742c72 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:09.250015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762c27b-26db-4954-8c9f-20eecd742c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:09.252310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762c27b-26db-4954-8c9f-20eecd742c72 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:09.406395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 896/1819] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:08 2022] POST /identity/v3/users => generated 286 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:09.407725 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af6fe8-3856-4e45-a291-fbf944c0eaf8 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:09.408043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af6fe8-3856-4e45-a291-fbf944c0eaf8 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:09.408386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af6fe8-3856-4e45-a291-fbf944c0eaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:09.442632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-54af6fe8-3856-4e45-a291-fbf944c0eaf8 None None] MFA Rules not processed for user `c29a91e099ba4cd5886fd910d978fb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:09.452274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-54af6fe8-3856-4e45-a291-fbf944c0eaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:09.470197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053832a-2d0d-4651-add2-7137a78a1793 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:09.480357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc39e9-4e59-4fad-8d6c-b5ccf56dd421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:09.481658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc39e9-4e59-4fad-8d6c-b5ccf56dd421 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:09.482121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc39e9-4e59-4fad-8d6c-b5ccf56dd421 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:09.482641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc39e9-4e59-4fad-8d6c-b5ccf56dd421 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:09.483916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc39e9-4e59-4fad-8d6c-b5ccf56dd421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:09.494994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc39e9-4e59-4fad-8d6c-b5ccf56dd421 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:09.605610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 897/1820] 10.222.0.30 () {58 vars in 1015 bytes} [Mon Aug 22 17:27:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:09.607846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af6fe8-3856-4e45-a291-fbf944c0eaf8 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:09.624660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 924/1821] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:09 2022] POST /identity/v3/projects => generated 403 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:09.630371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc39e9-4e59-4fad-8d6c-b5ccf56dd421 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:09.631973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af6fe8-3856-4e45-a291-fbf944c0eaf8 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:09.765815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf3392-8fc5-40f4-81b1-fc3f5d6017e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:09.769356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf3392-8fc5-40f4-81b1-fc3f5d6017e1 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:09.769786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf3392-8fc5-40f4-81b1-fc3f5d6017e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:09.784265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf3392-8fc5-40f4-81b1-fc3f5d6017e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:09.784265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf3392-8fc5-40f4-81b1-fc3f5d6017e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:09.784265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf3392-8fc5-40f4-81b1-fc3f5d6017e1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:09.784551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc39e9-4e59-4fad-8d6c-b5ccf56dd421 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:09.793302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 898/1822] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:09.801289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf3392-8fc5-40f4-81b1-fc3f5d6017e1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:09.809381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdbd08-cce9-4ffd-b977-ba00cec95b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:09.810455 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdbd08-cce9-4ffd-b977-ba00cec95b03 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:09.810988 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdbd08-cce9-4ffd-b977-ba00cec95b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:09.811846 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdbd08-cce9-4ffd-b977-ba00cec95b03 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:09.813094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdbd08-cce9-4ffd-b977-ba00cec95b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:09.822044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdbd08-cce9-4ffd-b977-ba00cec95b03 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:09.829582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf3392-8fc5-40f4-81b1-fc3f5d6017e1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:09.841074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bf1d3-671e-4d38-80ee-41960a6056fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:09.841883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bf1d3-671e-4d38-80ee-41960a6056fc admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:09.842271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bf1d3-671e-4d38-80ee-41960a6056fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:09.842720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bf1d3-671e-4d38-80ee-41960a6056fc admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:09.843493 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bf1d3-671e-4d38-80ee-41960a6056fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:09.845460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bf1d3-671e-4d38-80ee-41960a6056fc admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:09.894912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 899/1823] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:09.898099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bf1d3-671e-4d38-80ee-41960a6056fc admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:09.916124 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bf1d3-671e-4d38-80ee-41960a6056fc admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:09.926870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc457d-8fb4-4e5b-b32f-7a575ce3a993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:09.947500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc457d-8fb4-4e5b-b32f-7a575ce3a993 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:09.950279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc457d-8fb4-4e5b-b32f-7a575ce3a993 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:09.950815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc457d-8fb4-4e5b-b32f-7a575ce3a993 admin admin] PATH_INFO: `/v3/projects/efb9b57f8f324d4a92d4a311d13009a4/users/f3e34404618b4f32b5021aa5a2fec7f1/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:09.977262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc457d-8fb4-4e5b-b32f-7a575ce3a993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb9b57f8f324d4a92d4a311d13009a4, user_id=f3e34404618b4f32b5021aa5a2fec7f1, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:09.982306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc457d-8fb4-4e5b-b32f-7a575ce3a993 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:10.027264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 925/1824] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:09 2022] POST /identity/v3/users => generated 308 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:10.034330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdbd08-cce9-4ffd-b977-ba00cec95b03 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:10.064448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdbd08-cce9-4ffd-b977-ba00cec95b03 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:10.072105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dccef-9809-45f0-888b-2bf5949aca78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:10.073168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dccef-9809-45f0-888b-2bf5949aca78 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:10.073653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dccef-9809-45f0-888b-2bf5949aca78 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:10.074127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dccef-9809-45f0-888b-2bf5949aca78 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:10.075433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dccef-9809-45f0-888b-2bf5949aca78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:10.078350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dccef-9809-45f0-888b-2bf5949aca78 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:10.109526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 900/1825] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:09 2022] PUT /identity/v3/projects/efb9b57f8f324d4a92d4a311d13009a4/users/f3e34404618b4f32b5021aa5a2fec7f1/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:10.119486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42029f-7d87-49ff-b3af-2e80794c580a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:10.119486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42029f-7d87-49ff-b3af-2e80794c580a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:10.119889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42029f-7d87-49ff-b3af-2e80794c580a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:10.135799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 926/1826] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:10.143901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dccef-9809-45f0-888b-2bf5949aca78 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:10.167443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dccef-9809-45f0-888b-2bf5949aca78 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:10.175010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2e42029f-7d87-49ff-b3af-2e80794c580a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:10.176785 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2e42029f-7d87-49ff-b3af-2e80794c580a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:10.338925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ada30-7330-4f97-9f98-2e12d13b1828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:10.338925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ada30-7330-4f97-9f98-2e12d13b1828 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:10.338925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ada30-7330-4f97-9f98-2e12d13b1828 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:10.338925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ada30-7330-4f97-9f98-2e12d13b1828 admin admin] PATH_INFO: `/v3/projects/5b27fe866d0e4cd19c6c036f768d6c65/users/88fb00cfbaf7495a8b5f37e2fd1a1179/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:10.342041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 901/1827] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:10.360021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42029f-7d87-49ff-b3af-2e80794c580a None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:10.362279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2e42029f-7d87-49ff-b3af-2e80794c580a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:10.384880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ada30-7330-4f97-9f98-2e12d13b1828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b27fe866d0e4cd19c6c036f768d6c65, user_id=88fb00cfbaf7495a8b5f37e2fd1a1179, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:10.403265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ada30-7330-4f97-9f98-2e12d13b1828 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:10.571255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42029f-7d87-49ff-b3af-2e80794c580a None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:10.578036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d6d2a-d8ce-49ae-9500-05e625e5d1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:10.579743 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d6d2a-d8ce-49ae-9500-05e625e5d1ba service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:10.585827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d6d2a-d8ce-49ae-9500-05e625e5d1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:10.586804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d6d2a-d8ce-49ae-9500-05e625e5d1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:10.604797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d6d2a-d8ce-49ae-9500-05e625e5d1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:10.607517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d6d2a-d8ce-49ae-9500-05e625e5d1ba service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:10.635154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 902/1828] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:27:10.703946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 927/1829] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:10 2022] PUT /identity/v3/projects/5b27fe866d0e4cd19c6c036f768d6c65/users/88fb00cfbaf7495a8b5f37e2fd1a1179/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:10.708812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d6d2a-d8ce-49ae-9500-05e625e5d1ba service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:10.736643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d6d2a-d8ce-49ae-9500-05e625e5d1ba service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:10.852665 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:10.856116 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3faf5-cc78-44d0-9a6d-1821b687159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:10.856590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3faf5-cc78-44d0-9a6d-1821b687159a admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:10.856867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3faf5-cc78-44d0-9a6d-1821b687159a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:10.857140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3faf5-cc78-44d0-9a6d-1821b687159a admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:10.858645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3faf5-cc78-44d0-9a6d-1821b687159a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:10.859733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3faf5-cc78-44d0-9a6d-1821b687159a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:10.894876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 903/1830] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:10.905150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ada30-7330-4f97-9f98-2e12d13b1828 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:10.936203 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ada30-7330-4f97-9f98-2e12d13b1828 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:10.943576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb152a-b53b-4e45-87d0-5f436306e02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:10.945876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb152a-b53b-4e45-87d0-5f436306e02f admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:10.946188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb152a-b53b-4e45-87d0-5f436306e02f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:10.946956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb152a-b53b-4e45-87d0-5f436306e02f admin admin] PATH_INFO: `/v3/projects/5b27fe866d0e4cd19c6c036f768d6c65/users/88fb00cfbaf7495a8b5f37e2fd1a1179/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:10.950388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb152a-b53b-4e45-87d0-5f436306e02f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b27fe866d0e4cd19c6c036f768d6c65, user_id=88fb00cfbaf7495a8b5f37e2fd1a1179, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:10.966254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb152a-b53b-4e45-87d0-5f436306e02f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:10.983467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 928/1831] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:10 2022] PUT /identity/v3/projects/5b27fe866d0e4cd19c6c036f768d6c65/users/88fb00cfbaf7495a8b5f37e2fd1a1179/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:11.523732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3faf5-cc78-44d0-9a6d-1821b687159a admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:11.535438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb152a-b53b-4e45-87d0-5f436306e02f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:11.870621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb152a-b53b-4e45-87d0-5f436306e02f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:11.871010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3faf5-cc78-44d0-9a6d-1821b687159a admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:11.875649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb8e0b-d092-41d5-a86f-f66f5b9cfc22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:11.876982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8e0b-d092-41d5-a86f-f66f5b9cfc22 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:11.877582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8e0b-d092-41d5-a86f-f66f5b9cfc22 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:11.878400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8e0b-d092-41d5-a86f-f66f5b9cfc22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:11.888427 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc944c5-e61f-4545-ba7c-3504f8c16c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:11.889078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc944c5-e61f-4545-ba7c-3504f8c16c53 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:11.889611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc944c5-e61f-4545-ba7c-3504f8c16c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:11.889883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc944c5-e61f-4545-ba7c-3504f8c16c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:11.904341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb8e0b-d092-41d5-a86f-f66f5b9cfc22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:11.907267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb8e0b-d092-41d5-a86f-f66f5b9cfc22 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:11.913755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc944c5-e61f-4545-ba7c-3504f8c16c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:11.915437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc944c5-e61f-4545-ba7c-3504f8c16c53 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:11.931086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 904/1832] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:11.931086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc944c5-e61f-4545-ba7c-3504f8c16c53 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:11.931552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-abc944c5-e61f-4545-ba7c-3504f8c16c53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:11.944842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc944c5-e61f-4545-ba7c-3504f8c16c53 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:11.949263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 929/1833] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:11.949459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00143b-5c72-43c6-9e5f-c70cec855196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:11.950232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00143b-5c72-43c6-9e5f-c70cec855196 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:11.950537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00143b-5c72-43c6-9e5f-c70cec855196 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:11.950852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00143b-5c72-43c6-9e5f-c70cec855196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:11.951935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb8e0b-d092-41d5-a86f-f66f5b9cfc22 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:11.960605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00143b-5c72-43c6-9e5f-c70cec855196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:11.962879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00143b-5c72-43c6-9e5f-c70cec855196 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:11.970934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb8e0b-d092-41d5-a86f-f66f5b9cfc22 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:11.975103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 905/1834] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:11.976042 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00143b-5c72-43c6-9e5f-c70cec855196 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:11.983091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d5b6c-c830-4b0d-8354-393317b024c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:11.983829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d5b6c-c830-4b0d-8354-393317b024c7 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:11.984122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d5b6c-c830-4b0d-8354-393317b024c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:11.984415 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d5b6c-c830-4b0d-8354-393317b024c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:11.989188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00143b-5c72-43c6-9e5f-c70cec855196 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:12.002138 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c80ed-60e6-4152-aa7d-c0c348e6a304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:12.003486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c80ed-60e6-4152-aa7d-c0c348e6a304 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:12.003704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c80ed-60e6-4152-aa7d-c0c348e6a304 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:12.003847 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c80ed-60e6-4152-aa7d-c0c348e6a304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:12.016178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d5b6c-c830-4b0d-8354-393317b024c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:12.016178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d5b6c-c830-4b0d-8354-393317b024c7 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:12.025269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c80ed-60e6-4152-aa7d-c0c348e6a304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:12.026823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c80ed-60e6-4152-aa7d-c0c348e6a304 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:12.033134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 930/1835] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:12.053800 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 906/1836] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:12.183064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:14.814316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:15.729307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e7c02-5f47-421e-a403-6b2b93598010 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:15.729594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e7c02-5f47-421e-a403-6b2b93598010 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:15.729872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e7c02-5f47-421e-a403-6b2b93598010 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:15.731707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 931/1837] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:27:15.757364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796527f-cf62-4dbb-b980-c8531c1376e4 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:15.757364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796527f-cf62-4dbb-b980-c8531c1376e4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:15.757364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796527f-cf62-4dbb-b980-c8531c1376e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:15.800641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-6796527f-cf62-4dbb-b980-c8531c1376e4 None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:15.807459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-6796527f-cf62-4dbb-b980-c8531c1376e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:15.822816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:15.953311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3c981-d728-42a5-b4d6-078a2ed591be None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:15.954362 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3c981-d728-42a5-b4d6-078a2ed591be None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:15.954812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3c981-d728-42a5-b4d6-078a2ed591be None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:15.960730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 932/1838] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:27:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:27:16.109274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 907/1839] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:27:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:16.255011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3c981-d728-42a5-b4d6-078a2ed591be None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:16.259024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-30a3c981-d728-42a5-b4d6-078a2ed591be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:16.310557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3c981-d728-42a5-b4d6-078a2ed591be None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:16.318844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e10de0-c73f-4e5a-a36a-1fcacd5d88f4 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:16.319466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e10de0-c73f-4e5a-a36a-1fcacd5d88f4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:16.320297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e10de0-c73f-4e5a-a36a-1fcacd5d88f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:16.324112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a108316-842f-4542-a4b8-6b0647364ea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:16.334821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108316-842f-4542-a4b8-6b0647364ea3 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:16.334821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108316-842f-4542-a4b8-6b0647364ea3 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:16.334821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108316-842f-4542-a4b8-6b0647364ea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:16.352017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108316-842f-4542-a4b8-6b0647364ea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:16.355437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108316-842f-4542-a4b8-6b0647364ea3 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:16.388193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 933/1840] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:16.419372 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-27e10de0-c73f-4e5a-a36a-1fcacd5d88f4 None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:16.430856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-27e10de0-c73f-4e5a-a36a-1fcacd5d88f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:16.455174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 908/1841] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:27:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:16.540294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757e16e-83d9-4cfe-9f57-33aab3e9d14b None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:16.540776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757e16e-83d9-4cfe-9f57-33aab3e9d14b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:16.541621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757e16e-83d9-4cfe-9f57-33aab3e9d14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:16.654343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e10de0-c73f-4e5a-a36a-1fcacd5d88f4 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:16.692981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e10de0-c73f-4e5a-a36a-1fcacd5d88f4 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:16.695819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-4757e16e-83d9-4cfe-9f57-33aab3e9d14b None None] MFA Rules not processed for user `f3e34404618b4f32b5021aa5a2fec7f1`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:16.700346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4757e16e-83d9-4cfe-9f57-33aab3e9d14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:16.727193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30242963-63b8-4378-b3ad-14b7ac7bd8d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:16.728407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30242963-63b8-4378-b3ad-14b7ac7bd8d2 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:16.729624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30242963-63b8-4378-b3ad-14b7ac7bd8d2 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:16.729624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30242963-63b8-4378-b3ad-14b7ac7bd8d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:16.781141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30242963-63b8-4378-b3ad-14b7ac7bd8d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:16.789033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30242963-63b8-4378-b3ad-14b7ac7bd8d2 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:16.823346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 909/1842] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:16.828136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8529c1-0cfe-4efc-bc39-b0ae9c23e043 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:16.829006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8529c1-0cfe-4efc-bc39-b0ae9c23e043 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:16.833646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8529c1-0cfe-4efc-bc39-b0ae9c23e043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:16.941445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2f8529c1-0cfe-4efc-bc39-b0ae9c23e043 None None] MFA Rules not processed for user `88fb00cfbaf7495a8b5f37e2fd1a1179`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:16.946087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2f8529c1-0cfe-4efc-bc39-b0ae9c23e043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:17.013633 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:17.023304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 934/1843] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:17.029675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757e16e-83d9-4cfe-9f57-33aab3e9d14b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:17.137404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 910/1844] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:16 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:17.251422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757e16e-83d9-4cfe-9f57-33aab3e9d14b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:17.275237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4d9f8-7a5b-4398-a5de-ee0a43de4ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:17.276513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d9f8-7a5b-4398-a5de-ee0a43de4ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:17.277152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d9f8-7a5b-4398-a5de-ee0a43de4ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:17.277776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d9f8-7a5b-4398-a5de-ee0a43de4ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:17.279179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4d9f8-7a5b-4398-a5de-ee0a43de4ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:17.288748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4d9f8-7a5b-4398-a5de-ee0a43de4ee3 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:17.504029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 935/1845] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:17 2022] POST /identity/v3/projects => generated 371 bytes in 473 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:17.507499 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8529c1-0cfe-4efc-bc39-b0ae9c23e043 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:17.527636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8529c1-0cfe-4efc-bc39-b0ae9c23e043 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:17.673275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14434fea-590c-4f1e-8910-2f1e8b3729b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:17.674191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14434fea-590c-4f1e-8910-2f1e8b3729b0 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:17.674469 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14434fea-590c-4f1e-8910-2f1e8b3729b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:17.674847 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14434fea-590c-4f1e-8910-2f1e8b3729b0 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:17.676688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14434fea-590c-4f1e-8910-2f1e8b3729b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:17.678637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14434fea-590c-4f1e-8910-2f1e8b3729b0 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:17.867019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:17.875506 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 911/1846] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:17 2022] POST /identity/v3/users => generated 292 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:17.881168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4d9f8-7a5b-4398-a5de-ee0a43de4ee3 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:17.894021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4d9f8-7a5b-4398-a5de-ee0a43de4ee3 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:17.901228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0e410-2437-4b50-9bdb-a068ffdf6622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:17.901897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0e410-2437-4b50-9bdb-a068ffdf6622 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:17.902175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0e410-2437-4b50-9bdb-a068ffdf6622 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:17.902449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0e410-2437-4b50-9bdb-a068ffdf6622 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:17.903137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0e410-2437-4b50-9bdb-a068ffdf6622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:17.904768 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0e410-2437-4b50-9bdb-a068ffdf6622 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:17.920016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 936/1847] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:27:17.926279 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14434fea-590c-4f1e-8910-2f1e8b3729b0 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:17.938188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14434fea-590c-4f1e-8910-2f1e8b3729b0 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:17.963587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05259623-ccec-40e6-9538-ae135622d3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:17.963587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259623-ccec-40e6-9538-ae135622d3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:17.963587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259623-ccec-40e6-9538-ae135622d3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:17.963587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259623-ccec-40e6-9538-ae135622d3f8 admin admin] PATH_INFO: `/v3/projects/2bb3baf24cdc43a48e2549ba3429ac27/users/9ad75420642d433bbf7b491302a3276c/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:17.997810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05259623-ccec-40e6-9538-ae135622d3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb3baf24cdc43a48e2549ba3429ac27, user_id=9ad75420642d433bbf7b491302a3276c, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:18.001799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05259623-ccec-40e6-9538-ae135622d3f8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:18.274253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 912/1848] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:17 2022] PUT /identity/v3/projects/2bb3baf24cdc43a48e2549ba3429ac27/users/9ad75420642d433bbf7b491302a3276c/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:18.279404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0e410-2437-4b50-9bdb-a068ffdf6622 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:18.295093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0e410-2437-4b50-9bdb-a068ffdf6622 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:18.369764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b0e41-7fc2-459f-976f-c8b402c92af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:18.370634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b0e41-7fc2-459f-976f-c8b402c92af6 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:18.370958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b0e41-7fc2-459f-976f-c8b402c92af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:18.371315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b0e41-7fc2-459f-976f-c8b402c92af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:18.372210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b0e41-7fc2-459f-976f-c8b402c92af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:18.373923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b0e41-7fc2-459f-976f-c8b402c92af6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:18.393615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 937/1849] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:18.393615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:18.396159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05259623-ccec-40e6-9538-ae135622d3f8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:18.417460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05259623-ccec-40e6-9538-ae135622d3f8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:18.425585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b864d4-3a7a-4ade-904a-ee2cb1820679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:18.426827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b864d4-3a7a-4ade-904a-ee2cb1820679 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:18.428108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b864d4-3a7a-4ade-904a-ee2cb1820679 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:18.428824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b864d4-3a7a-4ade-904a-ee2cb1820679 admin admin] PATH_INFO: `/v3/projects/2bb3baf24cdc43a48e2549ba3429ac27/users/9ad75420642d433bbf7b491302a3276c/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:18.435011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b864d4-3a7a-4ade-904a-ee2cb1820679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb3baf24cdc43a48e2549ba3429ac27, user_id=9ad75420642d433bbf7b491302a3276c, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:18.439848 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b864d4-3a7a-4ade-904a-ee2cb1820679 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:18.466215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 913/1850] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:18 2022] PUT /identity/v3/projects/2bb3baf24cdc43a48e2549ba3429ac27/users/9ad75420642d433bbf7b491302a3276c/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:18.551675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091df4d-0bc1-406c-95be-5fff45e7e49c None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:18.553008 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091df4d-0bc1-406c-95be-5fff45e7e49c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:18.553915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091df4d-0bc1-406c-95be-5fff45e7e49c None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:18.555756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 938/1851] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:27:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:27:18.566935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5790fb-bfe1-48ad-ad3d-5cf95e190c56 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:18.567440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5790fb-bfe1-48ad-ad3d-5cf95e190c56 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:18.568594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5790fb-bfe1-48ad-ad3d-5cf95e190c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:18.603785 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-4f5790fb-bfe1-48ad-ad3d-5cf95e190c56 None None] MFA Rules not processed for user `26b5021285c44db9b05f9d15ceaf797f`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:18.606589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4f5790fb-bfe1-48ad-ad3d-5cf95e190c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:18.802880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 914/1852] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:27:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:18.822886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091df4d-0bc1-406c-95be-5fff45e7e49c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:18.825951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2091df4d-0bc1-406c-95be-5fff45e7e49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:18.863267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091df4d-0bc1-406c-95be-5fff45e7e49c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:18.875505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279db9a-ae31-445b-84e9-36f85cff299b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:18.876745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279db9a-ae31-445b-84e9-36f85cff299b service placement] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:18.877610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279db9a-ae31-445b-84e9-36f85cff299b service placement] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:18.878164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279db9a-ae31-445b-84e9-36f85cff299b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:18.899103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279db9a-ae31-445b-84e9-36f85cff299b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:18.902603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279db9a-ae31-445b-84e9-36f85cff299b service placement] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:18.923747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 939/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:20.070517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98408bb9-88b3-4fc5-9fec-73722d29fdba None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:20.071146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98408bb9-88b3-4fc5-9fec-73722d29fdba None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:20.071707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98408bb9-88b3-4fc5-9fec-73722d29fdba None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:20.075348 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 915/1854] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:27:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:27:20.089513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0c9dc-0702-4250-ab29-83555dda545b None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:20.090657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0c9dc-0702-4250-ab29-83555dda545b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:20.091205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0c9dc-0702-4250-ab29-83555dda545b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:20.114092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-ede0c9dc-0702-4250-ab29-83555dda545b None None] MFA Rules not processed for user `26b5021285c44db9b05f9d15ceaf797f`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:20.115743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ede0c9dc-0702-4250-ab29-83555dda545b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:20.126413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 940/1855] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:27:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:20.193415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98408bb9-88b3-4fc5-9fec-73722d29fdba None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:20.244205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98408bb9-88b3-4fc5-9fec-73722d29fdba None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:20.258703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66857d-99f6-4b5d-b35a-369fc63c86e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:20.262243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66857d-99f6-4b5d-b35a-369fc63c86e5 service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:20.263375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66857d-99f6-4b5d-b35a-369fc63c86e5 service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:20.265382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66857d-99f6-4b5d-b35a-369fc63c86e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:20.289496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66857d-99f6-4b5d-b35a-369fc63c86e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:20.294140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66857d-99f6-4b5d-b35a-369fc63c86e5 service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:20.305928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 916/1856] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:20.438605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0c9dc-0702-4250-ab29-83555dda545b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:20.684573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0c9dc-0702-4250-ab29-83555dda545b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:20.701287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:20.703351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:20.704098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:20.705048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] PATH_INFO: `/v3/users/0b2a968526f443a2be990b48bf18c503` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:20.708936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b2a968526f443a2be990b48bf18c503)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:20.713780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:20.997638 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b2a968526f443a2be990b48bf18c503'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:20.998577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b2a968526f443a2be990b48bf18c503'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:20.999481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b2a968526f443a2be990b48bf18c503'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:21.088892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b2a968526f443a2be990b48bf18c503'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:21.129832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b2a968526f443a2be990b48bf18c503'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:21.175788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 941/1857] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:20 2022] DELETE /identity/v3/users/0b2a968526f443a2be990b48bf18c503 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:21.524383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de611-2075-4766-a488-2ae4f468d40d None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:21.525239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de611-2075-4766-a488-2ae4f468d40d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:21.525394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de611-2075-4766-a488-2ae4f468d40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:21.700514 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3d5de611-2075-4766-a488-2ae4f468d40d None None] MFA Rules not processed for user `9ad75420642d433bbf7b491302a3276c`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:21.707046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3d5de611-2075-4766-a488-2ae4f468d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:22.077539 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 917/1858] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:22.103541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:22.108000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:22.177586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5de611-2075-4766-a488-2ae4f468d40d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:22.177586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3d5de611-2075-4766-a488-2ae4f468d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:22.253194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:22.352704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:22.378778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab187e-e39e-483b-999f-8d31dcdf76c0 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:22.399571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b270974-ea58-4cf8-bdf5-c2fb971309a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:22.401349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270974-ea58-4cf8-bdf5-c2fb971309a9 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:22.402088 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270974-ea58-4cf8-bdf5-c2fb971309a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:22.404349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270974-ea58-4cf8-bdf5-c2fb971309a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:22.408336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3b270974-ea58-4cf8-bdf5-c2fb971309a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:22.521830 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5de611-2075-4766-a488-2ae4f468d40d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:22.530663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:22.533031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:22.533888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:22.534893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] PATH_INFO: `/v3/projects/8c1c4f3e5c114af7b6fcbbebd9ee754c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:22.537805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c1c4f3e5c114af7b6fcbbebd9ee754c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:22.542110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:22.674387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b270974-ea58-4cf8-bdf5-c2fb971309a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:22.677984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b270974-ea58-4cf8-bdf5-c2fb971309a9 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:22.707141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 942/1859] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:22.708440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b270974-ea58-4cf8-bdf5-c2fb971309a9 service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:22.710721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3b270974-ea58-4cf8-bdf5-c2fb971309a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:22.773587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b270974-ea58-4cf8-bdf5-c2fb971309a9 service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:22.808803 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:22.809708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:22.821539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:22.823052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] PATH_INFO: `/v3/users/d0481f30d54544de82589fab75ad8593` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:22.826551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0481f30d54544de82589fab75ad8593)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:22.833765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:22.893746 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c1c4f3e5c114af7b6fcbbebd9ee754c', 'request_id': 'req-4fb54e03-4313-45aa-846c-923bdcc6b161'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:22.937921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c1c4f3e5c114af7b6fcbbebd9ee754c', 'request_id': 'req-4fb54e03-4313-45aa-846c-923bdcc6b161'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.111525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c1c4f3e5c114af7b6fcbbebd9ee754c', 'request_id': 'req-4fb54e03-4313-45aa-846c-923bdcc6b161'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.149905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] The token cache is being invalidate because project 8c1c4f3e5c114af7b6fcbbebd9ee754c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:23.150892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.152824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.160968 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 918/1860] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:22 2022] DELETE /identity/v3/projects/8c1c4f3e5c114af7b6fcbbebd9ee754c => generated 0 bytes in 987 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:23.167886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:23.176066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:23.269115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0481f30d54544de82589fab75ad8593'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.272229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0481f30d54544de82589fab75ad8593'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.277247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0481f30d54544de82589fab75ad8593'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.415348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0481f30d54544de82589fab75ad8593'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.421888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb54e03-4313-45aa-846c-923bdcc6b161 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:23.432126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:23.433220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:23.433998 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:23.434769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] PATH_INFO: `/v3/users/88fb00cfbaf7495a8b5f37e2fd1a1179` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:23.438072 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88fb00cfbaf7495a8b5f37e2fd1a1179)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:23.443916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:23.454719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-69974e8a-569c-4472-9240-c6cf15c548e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0481f30d54544de82589fab75ad8593'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.538310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 943/1861] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:22 2022] DELETE /identity/v3/users/d0481f30d54544de82589fab75ad8593 => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:23.552830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f59cf-2f4a-450e-91c8-3172c51d1e62 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:23.555074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f59cf-2f4a-450e-91c8-3172c51d1e62 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:23.556733 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f59cf-2f4a-450e-91c8-3172c51d1e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:23.601724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-332f59cf-2f4a-450e-91c8-3172c51d1e62 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:23.604122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-332f59cf-2f4a-450e-91c8-3172c51d1e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:23.706126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:23.761918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 944/1862] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:23.784274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88fb00cfbaf7495a8b5f37e2fd1a1179'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.785277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88fb00cfbaf7495a8b5f37e2fd1a1179'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.787474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88fb00cfbaf7495a8b5f37e2fd1a1179'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.810312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88fb00cfbaf7495a8b5f37e2fd1a1179'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.833269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:23.853107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88fb00cfbaf7495a8b5f37e2fd1a1179'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:23.900541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f59cf-2f4a-450e-91c8-3172c51d1e62 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:23.902500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-332f59cf-2f4a-450e-91c8-3172c51d1e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:24.046221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 919/1863] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:23 2022] DELETE /identity/v3/users/88fb00cfbaf7495a8b5f37e2fd1a1179 => generated 0 bytes in 881 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:24.063370 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f59cf-2f4a-450e-91c8-3172c51d1e62 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:24.251200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef03bca-a72f-42ac-a94d-acc7ed6356c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:24.251964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef03bca-a72f-42ac-a94d-acc7ed6356c2 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:24.252356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef03bca-a72f-42ac-a94d-acc7ed6356c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:24.252738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef03bca-a72f-42ac-a94d-acc7ed6356c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:24.258152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6ef03bca-a72f-42ac-a94d-acc7ed6356c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:24.454616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef03bca-a72f-42ac-a94d-acc7ed6356c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:24.456857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef03bca-a72f-42ac-a94d-acc7ed6356c2 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:24.479175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 945/1864] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:25.034366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:25.053854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:25.091889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d811d5f-9d47-46eb-a0b3-a9c4b9cffca3 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:25.100623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:25.101759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:25.102248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:25.103115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] PATH_INFO: `/v3/projects/5b27fe866d0e4cd19c6c036f768d6c65` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:25.105314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b27fe866d0e4cd19c6c036f768d6c65)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:25.109290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:25.165293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef03bca-a72f-42ac-a94d-acc7ed6356c2 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:25.167569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6ef03bca-a72f-42ac-a94d-acc7ed6356c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:25.189563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef03bca-a72f-42ac-a94d-acc7ed6356c2 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:25.199552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:25.200659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:25.201158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:25.201724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] PATH_INFO: `/v3/projects/ddfb94fcd2924c39b0268fcb0a89cd8b` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:25.204087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddfb94fcd2924c39b0268fcb0a89cd8b)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:25.207875 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:25.296245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b27fe866d0e4cd19c6c036f768d6c65', 'request_id': 'req-7aa5a835-4832-40ea-904f-54881a56c609'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:25.323586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b27fe866d0e4cd19c6c036f768d6c65', 'request_id': 'req-7aa5a835-4832-40ea-904f-54881a56c609'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:25.368009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddfb94fcd2924c39b0268fcb0a89cd8b', 'request_id': 'req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:25.369818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:25.377423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddfb94fcd2924c39b0268fcb0a89cd8b', 'request_id': 'req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:25.388062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddfb94fcd2924c39b0268fcb0a89cd8b', 'request_id': 'req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:25.419135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b27fe866d0e4cd19c6c036f768d6c65', 'request_id': 'req-7aa5a835-4832-40ea-904f-54881a56c609'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:25.432039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] The token cache is being invalidate because project 5b27fe866d0e4cd19c6c036f768d6c65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:25.432356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:25.433031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7aa5a835-4832-40ea-904f-54881a56c609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:25.434859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 920/1865] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:25 2022] DELETE /identity/v3/projects/5b27fe866d0e4cd19c6c036f768d6c65 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:25.441382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f8206-eed6-4a70-8437-219f32b1db96 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:25.441657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f8206-eed6-4a70-8437-219f32b1db96 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:25.442168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f8206-eed6-4a70-8437-219f32b1db96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:25.467571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b26f8206-eed6-4a70-8437-219f32b1db96 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:25.469311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b26f8206-eed6-4a70-8437-219f32b1db96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:25.483363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] The token cache is being invalidate because project ddfb94fcd2924c39b0268fcb0a89cd8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:25.486173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:25.489400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:25.495498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 946/1866] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:25 2022] DELETE /identity/v3/projects/ddfb94fcd2924c39b0268fcb0a89cd8b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:25.502377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:25.505639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:25.602706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 921/1867] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:25.606371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e6d50-aa7d-4235-9ff8-aacc3afe26a0 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:25.611792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:25.612115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503a0dd-c37d-41de-a4a3-9c3d5130c6bd None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:25.612422 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503a0dd-c37d-41de-a4a3-9c3d5130c6bd None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:25.612586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:25.612756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:25.612904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503a0dd-c37d-41de-a4a3-9c3d5130c6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:25.613049 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] PATH_INFO: `/v3/users/aaae4c51606e454f9836f3f84411fcaa` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:25.614208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaae4c51606e454f9836f3f84411fcaa)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:25.616883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:25.849789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-4503a0dd-c37d-41de-a4a3-9c3d5130c6bd None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:25.849789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4503a0dd-c37d-41de-a4a3-9c3d5130c6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:25.849789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 922/1868] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:25.849789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503a0dd-c37d-41de-a4a3-9c3d5130c6bd None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:25.849789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503a0dd-c37d-41de-a4a3-9c3d5130c6bd None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:25.850875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f5d72-ec06-4a7a-a5ca-3fc255949728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:25.850875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f5d72-ec06-4a7a-a5ca-3fc255949728 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:25.850875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f5d72-ec06-4a7a-a5ca-3fc255949728 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:25.850875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f5d72-ec06-4a7a-a5ca-3fc255949728 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:25.851388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f5d72-ec06-4a7a-a5ca-3fc255949728 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:27:25.851388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f5d72-ec06-4a7a-a5ca-3fc255949728 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:25.851388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f5d72-ec06-4a7a-a5ca-3fc255949728 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:25.851388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 923/1869] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:27:25.851388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f5d72-ec06-4a7a-a5ca-3fc255949728 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:25.851388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f5d72-ec06-4a7a-a5ca-3fc255949728 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:25.851959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee996-dd50-48d8-a098-ae97486ae314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:25.851959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee996-dd50-48d8-a098-ae97486ae314 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:25.851959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee996-dd50-48d8-a098-ae97486ae314 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:25.851959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee996-dd50-48d8-a098-ae97486ae314 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:25.852444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ee996-dd50-48d8-a098-ae97486ae314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:25.852444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ee996-dd50-48d8-a098-ae97486ae314 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:25.954904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 924/1870] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:25 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:27:25.961909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee996-dd50-48d8-a098-ae97486ae314 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:25.995804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:26.019762 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaae4c51606e454f9836f3f84411fcaa'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:26.020653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaae4c51606e454f9836f3f84411fcaa'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:26.021443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaae4c51606e454f9836f3f84411fcaa'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:26.058355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee996-dd50-48d8-a098-ae97486ae314 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:26.063657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9119c89-61f7-4cf3-b36d-5fb55291a094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:26.064304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9119c89-61f7-4cf3-b36d-5fb55291a094 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:26.064582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9119c89-61f7-4cf3-b36d-5fb55291a094 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:26.064851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9119c89-61f7-4cf3-b36d-5fb55291a094 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:26.065523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9119c89-61f7-4cf3-b36d-5fb55291a094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:26.068111 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9119c89-61f7-4cf3-b36d-5fb55291a094 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:26.105214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaae4c51606e454f9836f3f84411fcaa'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:26.116543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaae4c51606e454f9836f3f84411fcaa'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:26.139097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 947/1871] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:25 2022] DELETE /identity/v3/users/aaae4c51606e454f9836f3f84411fcaa => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:26.186592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 925/1872] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:25 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:27:26.189024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:26.190877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:26.271764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:26.277153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78349c3a-7542-41a5-abbc-f27e57ef983e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:26.283520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee874bd8-8c2c-4e85-bb2f-230f58ff766d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:26.284385 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee874bd8-8c2c-4e85-bb2f-230f58ff766d admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:26.284761 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee874bd8-8c2c-4e85-bb2f-230f58ff766d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:26.285143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee874bd8-8c2c-4e85-bb2f-230f58ff766d admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:26.286008 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee874bd8-8c2c-4e85-bb2f-230f58ff766d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:26.289261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee874bd8-8c2c-4e85-bb2f-230f58ff766d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:26.302922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 948/1873] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:27:26.309733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9119c89-61f7-4cf3-b36d-5fb55291a094 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:26.320836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9119c89-61f7-4cf3-b36d-5fb55291a094 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:26.326026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d269f-9e98-48a5-ab70-d881e0bc23ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:26.332965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d269f-9e98-48a5-ab70-d881e0bc23ec admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:26.332965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d269f-9e98-48a5-ab70-d881e0bc23ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:26.332965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d269f-9e98-48a5-ab70-d881e0bc23ec admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:26.332965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d269f-9e98-48a5-ab70-d881e0bc23ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:26.332965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d269f-9e98-48a5-ab70-d881e0bc23ec admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:26.346049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 926/1874] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:26.351628 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee874bd8-8c2c-4e85-bb2f-230f58ff766d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:26.379658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee874bd8-8c2c-4e85-bb2f-230f58ff766d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:26.398168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fb4a0-1838-4088-b6de-216c6310cbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:26.401765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fb4a0-1838-4088-b6de-216c6310cbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:26.401765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fb4a0-1838-4088-b6de-216c6310cbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:26.402515 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fb4a0-1838-4088-b6de-216c6310cbd5 admin admin] PATH_INFO: `/v3/projects/4039dcfd82b64b76b9ad2c88ff706959/users/52d3a940b0b34895ab4831f94245d3f3/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:26.436374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fb4a0-1838-4088-b6de-216c6310cbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4039dcfd82b64b76b9ad2c88ff706959, user_id=52d3a940b0b34895ab4831f94245d3f3, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:26.442229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fb4a0-1838-4088-b6de-216c6310cbd5 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:26.557229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 949/1875] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:26 2022] PUT /identity/v3/projects/4039dcfd82b64b76b9ad2c88ff706959/users/52d3a940b0b34895ab4831f94245d3f3/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:26.579633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6be6a0-1237-49ae-b720-66b5f95c6e37 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:26.580163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6be6a0-1237-49ae-b720-66b5f95c6e37 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:26.580654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6be6a0-1237-49ae-b720-66b5f95c6e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:26.656287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1c6be6a0-1237-49ae-b720-66b5f95c6e37 None None] MFA Rules not processed for user `52d3a940b0b34895ab4831f94245d3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:26.658373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1c6be6a0-1237-49ae-b720-66b5f95c6e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:26.713735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fb4a0-1838-4088-b6de-216c6310cbd5 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:26.717618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a83fb4a0-1838-4088-b6de-216c6310cbd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:26.817519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 927/1876] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:26.834003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6be6a0-1237-49ae-b720-66b5f95c6e37 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:26.836638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1c6be6a0-1237-49ae-b720-66b5f95c6e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:26.860961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fb4a0-1838-4088-b6de-216c6310cbd5 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:26.867304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:26.867978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:26.868371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:26.868657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] PATH_INFO: `/v3/projects/cf0b282744614428aa2e34957bb941e4` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:26.869917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0b282744614428aa2e34957bb941e4)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:26.871950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:26.955927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6be6a0-1237-49ae-b720-66b5f95c6e37 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:26.962982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb4d25-fe09-4023-b746-96c0074fc6ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:26.964030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb4d25-fe09-4023-b746-96c0074fc6ff service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:26.964507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb4d25-fe09-4023-b746-96c0074fc6ff service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:26.964992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb4d25-fe09-4023-b746-96c0074fc6ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:26.981502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb4d25-fe09-4023-b746-96c0074fc6ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:26.983903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb4d25-fe09-4023-b746-96c0074fc6ff service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:26.998507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 928/1877] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:27:27.001462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0b282744614428aa2e34957bb941e4', 'request_id': 'req-b54a468f-ace6-4c67-922c-664f0355ac65'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:27.011959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0b282744614428aa2e34957bb941e4', 'request_id': 'req-b54a468f-ace6-4c67-922c-664f0355ac65'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:27.021477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb4d25-fe09-4023-b746-96c0074fc6ff service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:27.026412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0b282744614428aa2e34957bb941e4', 'request_id': 'req-b54a468f-ace6-4c67-922c-664f0355ac65'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:27.034298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb4d25-fe09-4023-b746-96c0074fc6ff service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:27.079190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:27.135474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:27.136242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.136460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.136806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] PATH_INFO: `/v3/users/52d3a940b0b34895ab4831f94245d3f3` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.138063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d3a940b0b34895ab4831f94245d3f3)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:27.140762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:27.142182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] The token cache is being invalidate because project cf0b282744614428aa2e34957bb941e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:27.142613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:27.143557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b54a468f-ace6-4c67-922c-664f0355ac65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:27.146588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 950/1878] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:26 2022] DELETE /identity/v3/projects/cf0b282744614428aa2e34957bb941e4 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:27.152110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a026da-5442-42fc-91b8-a636aab91c39 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.152549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a026da-5442-42fc-91b8-a636aab91c39 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.152917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a026da-5442-42fc-91b8-a636aab91c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.179696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-14a026da-5442-42fc-91b8-a636aab91c39 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:27.181328 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-14a026da-5442-42fc-91b8-a636aab91c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:27.191881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 951/1879] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:27.201038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854b477-4290-467b-af33-2c3a722218c8 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.201570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854b477-4290-467b-af33-2c3a722218c8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.202033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854b477-4290-467b-af33-2c3a722218c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.226553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2854b477-4290-467b-af33-2c3a722218c8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:27.228102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2854b477-4290-467b-af33-2c3a722218c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:27.241980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 952/1880] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:27.246922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854b477-4290-467b-af33-2c3a722218c8 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:27.263913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854b477-4290-467b-af33-2c3a722218c8 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:27.273324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac37cc-5470-45dc-9e92-8316bbcb21c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:27.274457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac37cc-5470-45dc-9e92-8316bbcb21c4 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.275010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac37cc-5470-45dc-9e92-8316bbcb21c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.275497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac37cc-5470-45dc-9e92-8316bbcb21c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.276788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac37cc-5470-45dc-9e92-8316bbcb21c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:27:27.277462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac37cc-5470-45dc-9e92-8316bbcb21c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:27.280534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac37cc-5470-45dc-9e92-8316bbcb21c4 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:27.299495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 953/1881] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:27:27.305153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac37cc-5470-45dc-9e92-8316bbcb21c4 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:27.317511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac37cc-5470-45dc-9e92-8316bbcb21c4 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:27.325068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824271f-7440-4c64-9474-32f214164041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:27.325948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824271f-7440-4c64-9474-32f214164041 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.326625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824271f-7440-4c64-9474-32f214164041 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.326819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824271f-7440-4c64-9474-32f214164041 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.327843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824271f-7440-4c64-9474-32f214164041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:27.331148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824271f-7440-4c64-9474-32f214164041 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:27.356213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d3a940b0b34895ab4831f94245d3f3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:27.358318 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d3a940b0b34895ab4831f94245d3f3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:27.359302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d3a940b0b34895ab4831f94245d3f3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:27.371092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d3a940b0b34895ab4831f94245d3f3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:27.390127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-87412c42-c6c3-40bc-97d9-4834fed42726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d3a940b0b34895ab4831f94245d3f3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:27.423576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 954/1882] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:27 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:27:27.429080 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824271f-7440-4c64-9474-32f214164041 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:27.431807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7824271f-7440-4c64-9474-32f214164041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:27.450691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:27.511936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 929/1883] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:27 2022] DELETE /identity/v3/users/52d3a940b0b34895ab4831f94245d3f3 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:27.522883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598849-3907-4918-b800-95cf375a269e None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.523220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598849-3907-4918-b800-95cf375a269e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.523435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598849-3907-4918-b800-95cf375a269e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.528159 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824271f-7440-4c64-9474-32f214164041 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:27.546615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-cf598849-3907-4918-b800-95cf375a269e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:27.547961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cf598849-3907-4918-b800-95cf375a269e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:27.645478 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 930/1884] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:27.647301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4762c-d377-4111-8aec-24f6200bebf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:27.648587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4762c-d377-4111-8aec-24f6200bebf7 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.649309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4762c-d377-4111-8aec-24f6200bebf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.649774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4762c-d377-4111-8aec-24f6200bebf7 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.651157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4762c-d377-4111-8aec-24f6200bebf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:27.655944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4762c-d377-4111-8aec-24f6200bebf7 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:27.659332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598849-3907-4918-b800-95cf375a269e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:27.661181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cf598849-3907-4918-b800-95cf375a269e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:27.759608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598849-3907-4918-b800-95cf375a269e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:27.764026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc822b6-6839-4ffd-9508-8a178fc32f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:27.764945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc822b6-6839-4ffd-9508-8a178fc32f51 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.765212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc822b6-6839-4ffd-9508-8a178fc32f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.765486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc822b6-6839-4ffd-9508-8a178fc32f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.776199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:27.779287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc822b6-6839-4ffd-9508-8a178fc32f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:27.780090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 955/1885] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:27 2022] POST /identity/v3/users => generated 292 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:27.780891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc822b6-6839-4ffd-9508-8a178fc32f51 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:27.785094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4762c-d377-4111-8aec-24f6200bebf7 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:27.793753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 931/1886] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:27:27.803808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4762c-d377-4111-8aec-24f6200bebf7 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:27.810902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24400b1b-fba4-49d8-b06f-97fd25997823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:27.811910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24400b1b-fba4-49d8-b06f-97fd25997823 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.812354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24400b1b-fba4-49d8-b06f-97fd25997823 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.813008 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24400b1b-fba4-49d8-b06f-97fd25997823 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.814143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24400b1b-fba4-49d8-b06f-97fd25997823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:27.816497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24400b1b-fba4-49d8-b06f-97fd25997823 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:27.836379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 956/1887] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:27.838033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc822b6-6839-4ffd-9508-8a178fc32f51 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:27.854554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc822b6-6839-4ffd-9508-8a178fc32f51 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:27.863021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6146d-0408-45d0-ad3b-a2bd9b1a2535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:27.863863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6146d-0408-45d0-ad3b-a2bd9b1a2535 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.864159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6146d-0408-45d0-ad3b-a2bd9b1a2535 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.866735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6146d-0408-45d0-ad3b-a2bd9b1a2535 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.868503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6146d-0408-45d0-ad3b-a2bd9b1a2535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:27.870703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6146d-0408-45d0-ad3b-a2bd9b1a2535 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:27.890863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 932/1888] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:27:27.896149 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24400b1b-fba4-49d8-b06f-97fd25997823 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:27.925341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24400b1b-fba4-49d8-b06f-97fd25997823 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:27.933333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d2ce3-e43f-4a02-a23b-4677c7d2b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:27.934545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2ce3-e43f-4a02-a23b-4677c7d2b76b admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:27.935012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2ce3-e43f-4a02-a23b-4677c7d2b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:27.935439 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2ce3-e43f-4a02-a23b-4677c7d2b76b admin admin] PATH_INFO: `/v3/projects/646c9aa0fe82428ca4d65dd4e132ffde/users/0da9b6f63ffe4dcd89c7ce1f0918c506/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:27.959352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d2ce3-e43f-4a02-a23b-4677c7d2b76b admin admin] RBAC: Authorizing `identity:create_grant(project_id=646c9aa0fe82428ca4d65dd4e132ffde, user_id=0da9b6f63ffe4dcd89c7ce1f0918c506, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:27.963237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d2ce3-e43f-4a02-a23b-4677c7d2b76b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:28.118207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 957/1889] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:27 2022] PUT /identity/v3/projects/646c9aa0fe82428ca4d65dd4e132ffde/users/0da9b6f63ffe4dcd89c7ce1f0918c506/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:28.138994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdcb78-fdf1-47a9-b799-5dde180f7e52 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:28.140327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdcb78-fdf1-47a9-b799-5dde180f7e52 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:28.140811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdcb78-fdf1-47a9-b799-5dde180f7e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:28.207846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-f1bdcb78-fdf1-47a9-b799-5dde180f7e52 None None] MFA Rules not processed for user `0da9b6f63ffe4dcd89c7ce1f0918c506`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:28.209964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f1bdcb78-fdf1-47a9-b799-5dde180f7e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:28.324597 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 933/1890] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:28.330075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d2ce3-e43f-4a02-a23b-4677c7d2b76b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:28.461984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d2ce3-e43f-4a02-a23b-4677c7d2b76b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:28.475141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fd0c1-4b63-4cd8-a1cb-62de7cc5ba1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:28.475435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fd0c1-4b63-4cd8-a1cb-62de7cc5ba1a admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:28.475805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fd0c1-4b63-4cd8-a1cb-62de7cc5ba1a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:28.476302 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fd0c1-4b63-4cd8-a1cb-62de7cc5ba1a admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:28.477796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fd0c1-4b63-4cd8-a1cb-62de7cc5ba1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:28.481898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fd0c1-4b63-4cd8-a1cb-62de7cc5ba1a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:28.607811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdcb78-fdf1-47a9-b799-5dde180f7e52 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:28.610247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f1bdcb78-fdf1-47a9-b799-5dde180f7e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:28.642194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdcb78-fdf1-47a9-b799-5dde180f7e52 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:28.643320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:28.663264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:28.663264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:28.663264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:28.663264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] PATH_INFO: `/v3/projects/4039dcfd82b64b76b9ad2c88ff706959` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:28.663816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4039dcfd82b64b76b9ad2c88ff706959)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:28.663816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:28.678515 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 958/1891] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:28 2022] POST /identity/v3/projects => generated 383 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:28.686854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fd0c1-4b63-4cd8-a1cb-62de7cc5ba1a admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:28.820378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4039dcfd82b64b76b9ad2c88ff706959', 'request_id': 'req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:28.846919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fd0c1-4b63-4cd8-a1cb-62de7cc5ba1a admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:28.847222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4039dcfd82b64b76b9ad2c88ff706959', 'request_id': 'req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:28.929462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4039dcfd82b64b76b9ad2c88ff706959', 'request_id': 'req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:28.974856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] The token cache is being invalidate because project 4039dcfd82b64b76b9ad2c88ff706959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:28.975325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:28.976198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-8c66c6e5-e40e-47a0-9d3e-a09df2dc3e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:28.978722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 934/1892] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:28 2022] DELETE /identity/v3/projects/4039dcfd82b64b76b9ad2c88ff706959 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:28.986380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f592c-ce86-4c71-80ef-718da0fac944 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:28.986954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f592c-ce86-4c71-80ef-718da0fac944 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:28.987525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f592c-ce86-4c71-80ef-718da0fac944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.016884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411eb4a-352d-4c84-b1d0-2fee86bfd2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:29.017525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411eb4a-352d-4c84-b1d0-2fee86bfd2cf admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:29.017808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411eb4a-352d-4c84-b1d0-2fee86bfd2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:29.018085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411eb4a-352d-4c84-b1d0-2fee86bfd2cf admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.027943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411eb4a-352d-4c84-b1d0-2fee86bfd2cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:29.029100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411eb4a-352d-4c84-b1d0-2fee86bfd2cf admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:29.039044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-7e0f592c-ce86-4c71-80ef-718da0fac944 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:29.054577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7e0f592c-ce86-4c71-80ef-718da0fac944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:29.071320 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 935/1893] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:29.088115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9588d-c844-4290-a3aa-2bfcc69a47a8 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:29.092435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9588d-c844-4290-a3aa-2bfcc69a47a8 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:29.092435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9588d-c844-4290-a3aa-2bfcc69a47a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.137794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 959/1894] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:28 2022] POST /identity/v3/users => generated 298 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:29.142592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-12e9588d-c844-4290-a3aa-2bfcc69a47a8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:29.144014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-12e9588d-c844-4290-a3aa-2bfcc69a47a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:29.148354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411eb4a-352d-4c84-b1d0-2fee86bfd2cf admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:29.153167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4411eb4a-352d-4c84-b1d0-2fee86bfd2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:29.162539 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 936/1895] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:29.181547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9588d-c844-4290-a3aa-2bfcc69a47a8 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:29.214371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9588d-c844-4290-a3aa-2bfcc69a47a8 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:29.225403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411eb4a-352d-4c84-b1d0-2fee86bfd2cf admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:29.235888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a4c89-bf7d-4939-bc2b-e3a1bd7e36cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:29.242259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a4c89-bf7d-4939-bc2b-e3a1bd7e36cf admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:29.242890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a4c89-bf7d-4939-bc2b-e3a1bd7e36cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:29.243253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a4c89-bf7d-4939-bc2b-e3a1bd7e36cf admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.244455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a4c89-bf7d-4939-bc2b-e3a1bd7e36cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:27:29.244921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a4c89-bf7d-4939-bc2b-e3a1bd7e36cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:29.251557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3b073-8890-42fa-a9eb-530560ad7247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:29.252841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3b073-8890-42fa-a9eb-530560ad7247 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:29.253477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3b073-8890-42fa-a9eb-530560ad7247 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:29.255556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3b073-8890-42fa-a9eb-530560ad7247 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.257015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a4c89-bf7d-4939-bc2b-e3a1bd7e36cf admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:29.262769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3b073-8890-42fa-a9eb-530560ad7247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:29.266252 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3b073-8890-42fa-a9eb-530560ad7247 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:29.305135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 960/1896] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:29.311202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3b073-8890-42fa-a9eb-530560ad7247 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:29.322303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 937/1897] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:27:29.329367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a4c89-bf7d-4939-bc2b-e3a1bd7e36cf admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:29.330466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3b073-8890-42fa-a9eb-530560ad7247 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:29.356241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaed7fa-ca74-4311-940e-bdedbdba8e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:29.357247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed7fa-ca74-4311-940e-bdedbdba8e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:29.357618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed7fa-ca74-4311-940e-bdedbdba8e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:29.358194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed7fa-ca74-4311-940e-bdedbdba8e8a admin admin] PATH_INFO: `/v3/projects/87ae5cd8fb1e4a7dbc2e3bb4705924e3/users/d4651259a8314ef79ea36f602f3fc5af/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.370565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a4c89-bf7d-4939-bc2b-e3a1bd7e36cf admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:29.388326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaed7fa-ca74-4311-940e-bdedbdba8e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ae5cd8fb1e4a7dbc2e3bb4705924e3, user_id=d4651259a8314ef79ea36f602f3fc5af, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:29.390078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7331a-0b49-4623-b86c-29f71ab7077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:29.391765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7331a-0b49-4623-b86c-29f71ab7077e admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:29.395322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7331a-0b49-4623-b86c-29f71ab7077e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:29.395531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaed7fa-ca74-4311-940e-bdedbdba8e8a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:29.395759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7331a-0b49-4623-b86c-29f71ab7077e admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.395759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7331a-0b49-4623-b86c-29f71ab7077e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:29.401649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7331a-0b49-4623-b86c-29f71ab7077e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:29.541569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 961/1898] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:29 2022] PUT /identity/v3/projects/87ae5cd8fb1e4a7dbc2e3bb4705924e3/users/d4651259a8314ef79ea36f602f3fc5af/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:29.557675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaed7fa-ca74-4311-940e-bdedbdba8e8a admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:29.593455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 938/1899] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:29 2022] POST /identity/v3/projects => generated 385 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:27:29.597465 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7331a-0b49-4623-b86c-29f71ab7077e admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:29.616242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:29.682414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaed7fa-ca74-4311-940e-bdedbdba8e8a admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:29.688987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:29.753703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7331a-0b49-4623-b86c-29f71ab7077e admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:29.763952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56156e-9ff3-4bc4-a870-a7e2159b8372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:29.765503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56156e-9ff3-4bc4-a870-a7e2159b8372 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:29.766209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56156e-9ff3-4bc4-a870-a7e2159b8372 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:29.766707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56156e-9ff3-4bc4-a870-a7e2159b8372 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.768055 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56156e-9ff3-4bc4-a870-a7e2159b8372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:29.772696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56156e-9ff3-4bc4-a870-a7e2159b8372 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:29.816536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d4190-51aa-460d-af65-39396f2f75e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:29.817172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d4190-51aa-460d-af65-39396f2f75e2 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:29.817984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d4190-51aa-460d-af65-39396f2f75e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:29.818240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d4190-51aa-460d-af65-39396f2f75e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.820693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d4190-51aa-460d-af65-39396f2f75e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:29.821008 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d4190-51aa-460d-af65-39396f2f75e2 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:29.835349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 962/1900] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:29.843668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d4190-51aa-460d-af65-39396f2f75e2 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:29.852115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d4190-51aa-460d-af65-39396f2f75e2 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:29.857297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2af4fe-8ec2-4571-8d03-9bb8ca5c21f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:29.858216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af4fe-8ec2-4571-8d03-9bb8ca5c21f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:29.858520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af4fe-8ec2-4571-8d03-9bb8ca5c21f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:29.858837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af4fe-8ec2-4571-8d03-9bb8ca5c21f7 admin admin] PATH_INFO: `/v3/projects/87ae5cd8fb1e4a7dbc2e3bb4705924e3/users/d4651259a8314ef79ea36f602f3fc5af/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.861416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2af4fe-8ec2-4571-8d03-9bb8ca5c21f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ae5cd8fb1e4a7dbc2e3bb4705924e3, user_id=d4651259a8314ef79ea36f602f3fc5af, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:29.865504 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2af4fe-8ec2-4571-8d03-9bb8ca5c21f7 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:29.877850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 963/1901] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:29 2022] PUT /identity/v3/projects/87ae5cd8fb1e4a7dbc2e3bb4705924e3/users/d4651259a8314ef79ea36f602f3fc5af/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:29.889710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e498-cf41-4441-aa3c-41dece2f8f63 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:29.890067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e498-cf41-4441-aa3c-41dece2f8f63 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:29.890221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e498-cf41-4441-aa3c-41dece2f8f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:29.903793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 939/1902] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:29 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:29.905193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56156e-9ff3-4bc4-a870-a7e2159b8372 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:29.922452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56156e-9ff3-4bc4-a870-a7e2159b8372 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:29.923859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e903e498-cf41-4441-aa3c-41dece2f8f63 None None] MFA Rules not processed for user `d4651259a8314ef79ea36f602f3fc5af`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:29.925108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e903e498-cf41-4441-aa3c-41dece2f8f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:30.013522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa3bb-a3c1-4051-b99a-e872b81753fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:30.014771 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa3bb-a3c1-4051-b99a-e872b81753fc admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:30.015217 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa3bb-a3c1-4051-b99a-e872b81753fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:30.015654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa3bb-a3c1-4051-b99a-e872b81753fc admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:30.016765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fa3bb-a3c1-4051-b99a-e872b81753fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:30.019666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fa3bb-a3c1-4051-b99a-e872b81753fc admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:30.047410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 964/1903] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:30.054772 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 940/1904] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:30.058294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903e498-cf41-4441-aa3c-41dece2f8f63 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:30.063157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa3bb-a3c1-4051-b99a-e872b81753fc admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:30.065457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-889fa3bb-a3c1-4051-b99a-e872b81753fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:30.075870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903e498-cf41-4441-aa3c-41dece2f8f63 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:30.083837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b060577-d429-4268-8230-ee3ab932ea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:30.084716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b060577-d429-4268-8230-ee3ab932ea0c admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:30.085094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b060577-d429-4268-8230-ee3ab932ea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:30.085429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b060577-d429-4268-8230-ee3ab932ea0c admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:30.086266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b060577-d429-4268-8230-ee3ab932ea0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:30.088064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b060577-d429-4268-8230-ee3ab932ea0c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:30.102563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 965/1905] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:30.107416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b060577-d429-4268-8230-ee3ab932ea0c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:30.118689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b060577-d429-4268-8230-ee3ab932ea0c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:30.123896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842429c-b68a-45c1-95a2-66e37fd4a6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:30.124579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842429c-b68a-45c1-95a2-66e37fd4a6c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:30.124859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842429c-b68a-45c1-95a2-66e37fd4a6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:30.125134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842429c-b68a-45c1-95a2-66e37fd4a6c1 admin admin] PATH_INFO: `/v3/projects/8f2c7f3618004988ae2b62a1100a9f17/users/f371764cbe374a5fafb0d0b8e94d405c/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:30.144422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842429c-b68a-45c1-95a2-66e37fd4a6c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f2c7f3618004988ae2b62a1100a9f17, user_id=f371764cbe374a5fafb0d0b8e94d405c, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:30.149644 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842429c-b68a-45c1-95a2-66e37fd4a6c1 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:30.154542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa3bb-a3c1-4051-b99a-e872b81753fc admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:30.158402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b0628-fa1a-4b7f-950f-cb16dd1306ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:30.159249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b0628-fa1a-4b7f-950f-cb16dd1306ce service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:30.159533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b0628-fa1a-4b7f-950f-cb16dd1306ce service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:30.159802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b0628-fa1a-4b7f-950f-cb16dd1306ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:30.162345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ec6b0628-fa1a-4b7f-950f-cb16dd1306ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:30.222983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:30.240698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b0628-fa1a-4b7f-950f-cb16dd1306ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:30.257859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b0628-fa1a-4b7f-950f-cb16dd1306ce service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:30.258042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 966/1906] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:30 2022] PUT /identity/v3/projects/8f2c7f3618004988ae2b62a1100a9f17/users/f371764cbe374a5fafb0d0b8e94d405c/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:30.266612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534993c-083a-4e25-85b4-f6711234e29e None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:30.267047 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534993c-083a-4e25-85b4-f6711234e29e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:30.267418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534993c-083a-4e25-85b4-f6711234e29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:30.270855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 941/1907] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:27:30.313031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0534993c-083a-4e25-85b4-f6711234e29e None None] MFA Rules not processed for user `f371764cbe374a5fafb0d0b8e94d405c`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:30.314788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0534993c-083a-4e25-85b4-f6711234e29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:30.369410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:30.477571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 967/1908] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:30.486662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b0628-fa1a-4b7f-950f-cb16dd1306ce service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:30.516615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b0628-fa1a-4b7f-950f-cb16dd1306ce service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:30.658691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb6aa7-1db2-4a24-8d7e-cbcee87198a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:30.659901 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb6aa7-1db2-4a24-8d7e-cbcee87198a6 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:30.660679 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb6aa7-1db2-4a24-8d7e-cbcee87198a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:30.661194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb6aa7-1db2-4a24-8d7e-cbcee87198a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:30.663383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb6aa7-1db2-4a24-8d7e-cbcee87198a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:30.666611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb6aa7-1db2-4a24-8d7e-cbcee87198a6 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:30.892058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534993c-083a-4e25-85b4-f6711234e29e None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:30.892985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 942/1909] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:30 2022] POST /identity/v3/projects => generated 383 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:30.942890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534993c-083a-4e25-85b4-f6711234e29e None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:30.969121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb6aa7-1db2-4a24-8d7e-cbcee87198a6 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:30.970749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-81bb6aa7-1db2-4a24-8d7e-cbcee87198a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:31.089625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ab34d-2974-4039-9ae5-151bfcf2f5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:31.093107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ab34d-2974-4039-9ae5-151bfcf2f5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:31.093925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ab34d-2974-4039-9ae5-151bfcf2f5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:31.094013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ab34d-2974-4039-9ae5-151bfcf2f5f7 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:31.102136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ab34d-2974-4039-9ae5-151bfcf2f5f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:31.123110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ab34d-2974-4039-9ae5-151bfcf2f5f7 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:31.149582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb6aa7-1db2-4a24-8d7e-cbcee87198a6 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:31.166633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b98469-4126-48a9-ad89-b3dfe59632c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:31.167908 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b98469-4126-48a9-ad89-b3dfe59632c6 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:31.168071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b98469-4126-48a9-ad89-b3dfe59632c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:31.168580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b98469-4126-48a9-ad89-b3dfe59632c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:31.172848 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-15b98469-4126-48a9-ad89-b3dfe59632c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:31.231635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 968/1910] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:30 2022] POST /identity/v3/users => generated 292 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:31.240607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ab34d-2974-4039-9ae5-151bfcf2f5f7 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:31.266810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ab34d-2974-4039-9ae5-151bfcf2f5f7 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:31.276828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b98469-4126-48a9-ad89-b3dfe59632c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:31.278407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b98469-4126-48a9-ad89-b3dfe59632c6 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:31.280799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f7205-9d56-42ea-8888-0a8c04c77071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:31.282121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f7205-9d56-42ea-8888-0a8c04c77071 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:31.283165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f7205-9d56-42ea-8888-0a8c04c77071 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:31.283727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f7205-9d56-42ea-8888-0a8c04c77071 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:31.285344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f7205-9d56-42ea-8888-0a8c04c77071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:31.290117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f7205-9d56-42ea-8888-0a8c04c77071 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:31.291417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 943/1911] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:27:31.310738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 969/1912] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:31.316788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b98469-4126-48a9-ad89-b3dfe59632c6 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:31.328866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b98469-4126-48a9-ad89-b3dfe59632c6 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:31.336511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa2ff3-5f63-4d35-991c-76e7084473d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:31.337283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa2ff3-5f63-4d35-991c-76e7084473d0 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:31.337798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa2ff3-5f63-4d35-991c-76e7084473d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:31.338152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa2ff3-5f63-4d35-991c-76e7084473d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:31.339146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa2ff3-5f63-4d35-991c-76e7084473d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:31.341850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa2ff3-5f63-4d35-991c-76e7084473d0 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:31.369961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 944/1913] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:31.374608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f7205-9d56-42ea-8888-0a8c04c77071 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:31.392977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f7205-9d56-42ea-8888-0a8c04c77071 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:31.402154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bdfa4-d5f1-487e-a876-32e5455f5743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:31.403544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bdfa4-d5f1-487e-a876-32e5455f5743 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:31.404132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bdfa4-d5f1-487e-a876-32e5455f5743 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:31.404702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bdfa4-d5f1-487e-a876-32e5455f5743 admin admin] PATH_INFO: `/v3/projects/51d58cdb5d59490195696f6ea2a882de/users/89484b16dc0949fcb8868494b401b812/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:31.453050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bdfa4-d5f1-487e-a876-32e5455f5743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d58cdb5d59490195696f6ea2a882de, user_id=89484b16dc0949fcb8868494b401b812, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:31.455936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bdfa4-d5f1-487e-a876-32e5455f5743 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:31.582786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 970/1914] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:31 2022] PUT /identity/v3/projects/51d58cdb5d59490195696f6ea2a882de/users/89484b16dc0949fcb8868494b401b812/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:31.597606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e050a-56cc-4ba0-a921-b8ddf2f00fe2 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:31.597887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e050a-56cc-4ba0-a921-b8ddf2f00fe2 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:31.609985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e050a-56cc-4ba0-a921-b8ddf2f00fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:31.672888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:31.689839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-414e050a-56cc-4ba0-a921-b8ddf2f00fe2 None None] MFA Rules not processed for user `89484b16dc0949fcb8868494b401b812`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:31.692156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-414e050a-56cc-4ba0-a921-b8ddf2f00fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:31.820051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 945/1915] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:31.826152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bdfa4-d5f1-487e-a876-32e5455f5743 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:31.931340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bdfa4-d5f1-487e-a876-32e5455f5743 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:31.937570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012c634-d5ab-4d11-a276-494f5991b0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:31.938214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012c634-d5ab-4d11-a276-494f5991b0b6 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:31.938624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012c634-d5ab-4d11-a276-494f5991b0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:31.939007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012c634-d5ab-4d11-a276-494f5991b0b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:31.939761 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012c634-d5ab-4d11-a276-494f5991b0b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:31.941976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012c634-d5ab-4d11-a276-494f5991b0b6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:32.088089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 971/1916] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:31 2022] POST /identity/v3/projects => generated 385 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:32.092246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e050a-56cc-4ba0-a921-b8ddf2f00fe2 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:32.111482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e050a-56cc-4ba0-a921-b8ddf2f00fe2 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:32.189993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba76a0-f4e8-4e5a-98f9-6984d3138328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:32.190700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba76a0-f4e8-4e5a-98f9-6984d3138328 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:32.190835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba76a0-f4e8-4e5a-98f9-6984d3138328 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:32.191150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba76a0-f4e8-4e5a-98f9-6984d3138328 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:32.191853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba76a0-f4e8-4e5a-98f9-6984d3138328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:32.194455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba76a0-f4e8-4e5a-98f9-6984d3138328 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:32.324142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 946/1917] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:32 2022] POST /identity/v3/users => generated 299 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:32.328658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012c634-d5ab-4d11-a276-494f5991b0b6 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:32.343255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012c634-d5ab-4d11-a276-494f5991b0b6 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:32.350355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4513a-c638-4db5-aef9-f27246971faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:32.351460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4513a-c638-4db5-aef9-f27246971faa admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:32.351999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4513a-c638-4db5-aef9-f27246971faa admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:32.352498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4513a-c638-4db5-aef9-f27246971faa admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:32.353731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4513a-c638-4db5-aef9-f27246971faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:32.356979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4513a-c638-4db5-aef9-f27246971faa admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:32.361259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:32.379702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 972/1918] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:32.385262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba76a0-f4e8-4e5a-98f9-6984d3138328 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:32.399262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba76a0-f4e8-4e5a-98f9-6984d3138328 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:32.406739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e6fdb-1975-4949-ac6b-2db699eaaa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:32.407789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e6fdb-1975-4949-ac6b-2db699eaaa7e admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:32.408228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e6fdb-1975-4949-ac6b-2db699eaaa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:32.408673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e6fdb-1975-4949-ac6b-2db699eaaa7e admin admin] PATH_INFO: `/v3/projects/c041a33b44a54c7ab108fa84e48a65e4/users/bf81e3b8dffa4cffb178aa4b5fee2a04/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:32.436559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e6fdb-1975-4949-ac6b-2db699eaaa7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c041a33b44a54c7ab108fa84e48a65e4, user_id=bf81e3b8dffa4cffb178aa4b5fee2a04, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:32.441916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e6fdb-1975-4949-ac6b-2db699eaaa7e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:32.547947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 947/1919] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:32 2022] PUT /identity/v3/projects/c041a33b44a54c7ab108fa84e48a65e4/users/bf81e3b8dffa4cffb178aa4b5fee2a04/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:32.561993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4513a-c638-4db5-aef9-f27246971faa admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:32.577073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4513a-c638-4db5-aef9-f27246971faa admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:32.676146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98701a-f251-48c5-bb89-31a9e93944fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:32.677578 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98701a-f251-48c5-bb89-31a9e93944fa admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:32.678008 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98701a-f251-48c5-bb89-31a9e93944fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:32.678436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98701a-f251-48c5-bb89-31a9e93944fa admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:32.679389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98701a-f251-48c5-bb89-31a9e93944fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:32.681504 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98701a-f251-48c5-bb89-31a9e93944fa admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:32.698779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 973/1920] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:27:32.706722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e6fdb-1975-4949-ac6b-2db699eaaa7e admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:32.725312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e6fdb-1975-4949-ac6b-2db699eaaa7e admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:32.732012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f4d9a-b3d6-4ff4-9ebb-4825e048601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:32.733165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f4d9a-b3d6-4ff4-9ebb-4825e048601c admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:32.733761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f4d9a-b3d6-4ff4-9ebb-4825e048601c admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:32.734100 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f4d9a-b3d6-4ff4-9ebb-4825e048601c admin admin] PATH_INFO: `/v3/projects/c041a33b44a54c7ab108fa84e48a65e4/users/bf81e3b8dffa4cffb178aa4b5fee2a04/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:32.737901 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f4d9a-b3d6-4ff4-9ebb-4825e048601c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c041a33b44a54c7ab108fa84e48a65e4, user_id=bf81e3b8dffa4cffb178aa4b5fee2a04, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:32.744366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f4d9a-b3d6-4ff4-9ebb-4825e048601c admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:32.766767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 948/1921] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:32 2022] PUT /identity/v3/projects/c041a33b44a54c7ab108fa84e48a65e4/users/bf81e3b8dffa4cffb178aa4b5fee2a04/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:32.789108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17361760-5a0d-4aa0-957f-b04ef117f694 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:32.789633 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17361760-5a0d-4aa0-957f-b04ef117f694 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:32.790123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17361760-5a0d-4aa0-957f-b04ef117f694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:32.855656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-17361760-5a0d-4aa0-957f-b04ef117f694 None None] MFA Rules not processed for user `bf81e3b8dffa4cffb178aa4b5fee2a04`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:32.868494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-17361760-5a0d-4aa0-957f-b04ef117f694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:32.981138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 974/1922] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:32.990153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f4d9a-b3d6-4ff4-9ebb-4825e048601c admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:33.103050 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f4d9a-b3d6-4ff4-9ebb-4825e048601c admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:33.109315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553e326-ed33-4d4d-b587-1c87c365c3a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:33.110312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553e326-ed33-4d4d-b587-1c87c365c3a2 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:33.110808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553e326-ed33-4d4d-b587-1c87c365c3a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:33.111539 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553e326-ed33-4d4d-b587-1c87c365c3a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:33.125457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:33.215544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553e326-ed33-4d4d-b587-1c87c365c3a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:33.218418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553e326-ed33-4d4d-b587-1c87c365c3a2 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:33.238074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 949/1923] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:36.517518 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17361760-5a0d-4aa0-957f-b04ef117f694 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:36.520061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-17361760-5a0d-4aa0-957f-b04ef117f694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:36.667174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17361760-5a0d-4aa0-957f-b04ef117f694 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:36.672696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:36.673634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:36.673995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:36.674344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] PATH_INFO: `/v3/users/41d49d19fa8442fb855ac0a1926d8587` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:36.676058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d49d19fa8442fb855ac0a1926d8587)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:36.679660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:36.713358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:37.019575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d49d19fa8442fb855ac0a1926d8587'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:37.027186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d49d19fa8442fb855ac0a1926d8587'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:37.031289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d49d19fa8442fb855ac0a1926d8587'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:37.181861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d49d19fa8442fb855ac0a1926d8587'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:37.194466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d49d19fa8442fb855ac0a1926d8587'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:37.236274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 975/1924] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:36 2022] DELETE /identity/v3/users/41d49d19fa8442fb855ac0a1926d8587 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:37.243247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6e0c5-3f60-4c15-84c1-2cb0868e043e None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:37.243802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6e0c5-3f60-4c15-84c1-2cb0868e043e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:37.243802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6e0c5-3f60-4c15-84c1-2cb0868e043e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:37.283950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-74f6e0c5-3f60-4c15-84c1-2cb0868e043e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:37.285366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-74f6e0c5-3f60-4c15-84c1-2cb0868e043e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:37.385400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 950/1925] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:37.401893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:37.403663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:37.412623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6e0c5-3f60-4c15-84c1-2cb0868e043e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:37.414376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-74f6e0c5-3f60-4c15-84c1-2cb0868e043e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:37.502237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6e0c5-3f60-4c15-84c1-2cb0868e043e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:37.506640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3528dc4-deb0-4011-9412-5722c7706901 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:37.507514 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c4552-e1b5-401a-a085-ee3695238b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:37.508167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c4552-e1b5-401a-a085-ee3695238b12 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:37.508448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c4552-e1b5-401a-a085-ee3695238b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:37.508926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c4552-e1b5-401a-a085-ee3695238b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:37.511942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8f340-7cf8-4c9c-919b-567c6b4010df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:37.512534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8f340-7cf8-4c9c-919b-567c6b4010df service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:37.512809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8f340-7cf8-4c9c-919b-567c6b4010df service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:37.513136 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8f340-7cf8-4c9c-919b-567c6b4010df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:37.515258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7ff8f340-7cf8-4c9c-919b-567c6b4010df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:37.520159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c4552-e1b5-401a-a085-ee3695238b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:37.522062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c4552-e1b5-401a-a085-ee3695238b12 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:37.533867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 951/1926] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:37.660965 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8f340-7cf8-4c9c-919b-567c6b4010df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:37.663549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8f340-7cf8-4c9c-919b-567c6b4010df service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:37.677450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 976/1927] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:38.291523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c4552-e1b5-401a-a085-ee3695238b12 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:38.294226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3f9c4552-e1b5-401a-a085-ee3695238b12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:38.321798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c4552-e1b5-401a-a085-ee3695238b12 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:38.328189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:38.329433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:38.330005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:38.330700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] PATH_INFO: `/v3/projects/3ddb806c902742d1bc37ede524c27140` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:38.332550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ddb806c902742d1bc37ede524c27140)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:38.336371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:38.386619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403d262-117f-4354-8a5f-3f7bc46cdf3b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:38.388691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403d262-117f-4354-8a5f-3f7bc46cdf3b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:38.389871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403d262-117f-4354-8a5f-3f7bc46cdf3b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:38.394500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 977/1928] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:27:38 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:27:38.424907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f171f52-2cf8-4f87-9aa0-49c0750058d3 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:38.428235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f171f52-2cf8-4f87-9aa0-49c0750058d3 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:38.448367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f171f52-2cf8-4f87-9aa0-49c0750058d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:38.501420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0f171f52-2cf8-4f87-9aa0-49c0750058d3 None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:38.504120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0f171f52-2cf8-4f87-9aa0-49c0750058d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:38.558117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:38.564959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ddb806c902742d1bc37ede524c27140', 'request_id': 'req-f338ee74-27e5-465d-ba35-1009aa50885c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:38.578856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ddb806c902742d1bc37ede524c27140', 'request_id': 'req-f338ee74-27e5-465d-ba35-1009aa50885c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:38.646247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:38.682299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 978/1929] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:27:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:38.704069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ddb806c902742d1bc37ede524c27140', 'request_id': 'req-f338ee74-27e5-465d-ba35-1009aa50885c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:38.706505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f171f52-2cf8-4f87-9aa0-49c0750058d3 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:38.710470 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0f171f52-2cf8-4f87-9aa0-49c0750058d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:38.731443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f171f52-2cf8-4f87-9aa0-49c0750058d3 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:38.742038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] The token cache is being invalidate because project 3ddb806c902742d1bc37ede524c27140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:38.742618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:38.745945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:38.750475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 952/1930] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:38 2022] DELETE /identity/v3/projects/3ddb806c902742d1bc37ede524c27140 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:38.757308 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:38.760272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:38.882121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5c17f-3fbe-4c90-a3be-5d8e6cc95f70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:38.883482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5c17f-3fbe-4c90-a3be-5d8e6cc95f70 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:38.883945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5c17f-3fbe-4c90-a3be-5d8e6cc95f70 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:38.884383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5c17f-3fbe-4c90-a3be-5d8e6cc95f70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:38.888197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-19f5c17f-3fbe-4c90-a3be-5d8e6cc95f70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:38.912556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338ee74-27e5-465d-ba35-1009aa50885c admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:38.914879 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5c17f-3fbe-4c90-a3be-5d8e6cc95f70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:38.918310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5c17f-3fbe-4c90-a3be-5d8e6cc95f70 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:38.923393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:38.924811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:38.925383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:38.926106 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] PATH_INFO: `/v3/users/1380d42c1e634f678d1381988c8111c7` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:38.929847 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1380d42c1e634f678d1381988c8111c7)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:38.933873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:38.989061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 979/1931] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:39.260622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1380d42c1e634f678d1381988c8111c7'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:39.267184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1380d42c1e634f678d1381988c8111c7'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:39.267184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1380d42c1e634f678d1381988c8111c7'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:39.276397 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5c17f-3fbe-4c90-a3be-5d8e6cc95f70 service nova] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:39.278412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-19f5c17f-3fbe-4c90-a3be-5d8e6cc95f70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:39.278792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1380d42c1e634f678d1381988c8111c7'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:39.312823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1380d42c1e634f678d1381988c8111c7'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:39.399933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 953/1932] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:38 2022] DELETE /identity/v3/users/1380d42c1e634f678d1381988c8111c7 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:39.428305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5c17f-3fbe-4c90-a3be-5d8e6cc95f70 service nova] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:39.529293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e6361-5817-4bf9-8409-ea177f1fb5df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:39.529929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e6361-5817-4bf9-8409-ea177f1fb5df service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:39.530344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e6361-5817-4bf9-8409-ea177f1fb5df service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:39.530799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e6361-5817-4bf9-8409-ea177f1fb5df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:39.533108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-706e6361-5817-4bf9-8409-ea177f1fb5df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:39.595419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:39.647729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e6361-5817-4bf9-8409-ea177f1fb5df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:39.649239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e6361-5817-4bf9-8409-ea177f1fb5df service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:39.662824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 980/1933] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:40.039492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:40.041237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:40.055744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe847e-a208-4214-86de-a147ee9dd883 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:40.074265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc24f4-b5a0-4dfc-afda-a0634a38ab1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:40.077088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc24f4-b5a0-4dfc-afda-a0634a38ab1a service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:40.079043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc24f4-b5a0-4dfc-afda-a0634a38ab1a service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:40.079633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc24f4-b5a0-4dfc-afda-a0634a38ab1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:40.085458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c4cc24f4-b5a0-4dfc-afda-a0634a38ab1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:40.184952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:40.216125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc24f4-b5a0-4dfc-afda-a0634a38ab1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:40.219317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc24f4-b5a0-4dfc-afda-a0634a38ab1a service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:40.234839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e6361-5817-4bf9-8409-ea177f1fb5df service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:40.237112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-706e6361-5817-4bf9-8409-ea177f1fb5df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:40.258233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 954/1934] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:40.375064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e6361-5817-4bf9-8409-ea177f1fb5df service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:40.380139 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:40.380843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:40.381134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:40.381407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] PATH_INFO: `/v3/projects/d8b616ba3c9e426b92f6e41302d6c76c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:40.382696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b616ba3c9e426b92f6e41302d6c76c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:40.384711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:40.568602 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b616ba3c9e426b92f6e41302d6c76c', 'request_id': 'req-fe059a61-918e-49fe-969a-e3dc4d449490'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:40.577429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b616ba3c9e426b92f6e41302d6c76c', 'request_id': 'req-fe059a61-918e-49fe-969a-e3dc4d449490'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:40.586627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b616ba3c9e426b92f6e41302d6c76c', 'request_id': 'req-fe059a61-918e-49fe-969a-e3dc4d449490'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:40.815715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] The token cache is being invalidate because project d8b616ba3c9e426b92f6e41302d6c76c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:40.816376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:40.817585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fe059a61-918e-49fe-969a-e3dc4d449490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:40.819757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 981/1935] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:40 2022] DELETE /identity/v3/projects/d8b616ba3c9e426b92f6e41302d6c76c => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:40.842602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198e7ae-55a6-4e48-8791-17b6ca65f8a3 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:40.842602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198e7ae-55a6-4e48-8791-17b6ca65f8a3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:40.842602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198e7ae-55a6-4e48-8791-17b6ca65f8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:40.868058 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1198e7ae-55a6-4e48-8791-17b6ca65f8a3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:40.869625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1198e7ae-55a6-4e48-8791-17b6ca65f8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:40.973137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 955/1936] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:40.983159 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c9e9c-3e12-4e2c-b9b0-881e5930f192 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:40.983788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c9e9c-3e12-4e2c-b9b0-881e5930f192 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:40.984391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c9e9c-3e12-4e2c-b9b0-881e5930f192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:41.028349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-237c9e9c-3e12-4e2c-b9b0-881e5930f192 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:41.031315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-237c9e9c-3e12-4e2c-b9b0-881e5930f192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:41.051046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 982/1937] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:41.058064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198e7ae-55a6-4e48-8791-17b6ca65f8a3 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:41.083218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198e7ae-55a6-4e48-8791-17b6ca65f8a3 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:41.089685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a0f7c-4a37-4e58-b630-b2e4c1f9297e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:41.090430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a0f7c-4a37-4e58-b630-b2e4c1f9297e admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:41.090752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a0f7c-4a37-4e58-b630-b2e4c1f9297e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:41.091043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a0f7c-4a37-4e58-b630-b2e4c1f9297e admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:41.091704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a0f7c-4a37-4e58-b630-b2e4c1f9297e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:27:41.092017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a0f7c-4a37-4e58-b630-b2e4c1f9297e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:41.093933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a0f7c-4a37-4e58-b630-b2e4c1f9297e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:41.111712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 956/1938] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:27:41.117443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c9e9c-3e12-4e2c-b9b0-881e5930f192 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:41.151896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c9e9c-3e12-4e2c-b9b0-881e5930f192 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:41.169955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c0cc8-2f05-4a8d-b608-83dad205d5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:41.170782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c0cc8-2f05-4a8d-b608-83dad205d5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:41.171268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c0cc8-2f05-4a8d-b608-83dad205d5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:41.176230 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c0cc8-2f05-4a8d-b608-83dad205d5b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:41.178100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c0cc8-2f05-4a8d-b608-83dad205d5b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:41.182554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c0cc8-2f05-4a8d-b608-83dad205d5b2 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:41.436936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 983/1939] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:41 2022] POST /identity/v3/projects => generated 367 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:41.444967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a0f7c-4a37-4e58-b630-b2e4c1f9297e admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:41.457591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a0f7c-4a37-4e58-b630-b2e4c1f9297e admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:41.552861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e9be1-6414-41db-93d5-e0dbad2c1172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:41.552861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e9be1-6414-41db-93d5-e0dbad2c1172 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:41.552861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e9be1-6414-41db-93d5-e0dbad2c1172 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:41.552861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e9be1-6414-41db-93d5-e0dbad2c1172 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:41.553142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e9be1-6414-41db-93d5-e0dbad2c1172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:41.553142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e9be1-6414-41db-93d5-e0dbad2c1172 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:42.746395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 957/1940] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:41 2022] POST /identity/v3/users => generated 284 bytes in 1302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:42.768108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c0cc8-2f05-4a8d-b608-83dad205d5b2 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:42.778821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c0cc8-2f05-4a8d-b608-83dad205d5b2 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:42.783662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9c30d-6a38-40b0-ade3-e2a74ad3cf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:42.784283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c30d-6a38-40b0-ade3-e2a74ad3cf5d admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:42.784559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c30d-6a38-40b0-ade3-e2a74ad3cf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:42.784835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c30d-6a38-40b0-ade3-e2a74ad3cf5d admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:42.785481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9c30d-6a38-40b0-ade3-e2a74ad3cf5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:42.787813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9c30d-6a38-40b0-ade3-e2a74ad3cf5d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:42.815190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 984/1941] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:42.819916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e9be1-6414-41db-93d5-e0dbad2c1172 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:42.832825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e9be1-6414-41db-93d5-e0dbad2c1172 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:42.839632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f57520-35a0-4912-b407-5d3d2d638c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:42.840306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f57520-35a0-4912-b407-5d3d2d638c3f admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:42.840620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f57520-35a0-4912-b407-5d3d2d638c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:42.841020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f57520-35a0-4912-b407-5d3d2d638c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:42.841728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f57520-35a0-4912-b407-5d3d2d638c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:42.843574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f57520-35a0-4912-b407-5d3d2d638c3f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:42.860219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 958/1942] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:42.865235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9c30d-6a38-40b0-ade3-e2a74ad3cf5d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:42.875466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9c30d-6a38-40b0-ade3-e2a74ad3cf5d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:42.880646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cfdd9-234d-47d9-860b-b272589b810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:42.881279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cfdd9-234d-47d9-860b-b272589b810d admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:42.881591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cfdd9-234d-47d9-860b-b272589b810d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:42.881895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cfdd9-234d-47d9-860b-b272589b810d admin admin] PATH_INFO: `/v3/projects/d370081bc7cb45ecab9c396c9bd130ff/users/eda88161285a4fb5b4b72cd6f68bf4ec/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:42.900340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978cfdd9-234d-47d9-860b-b272589b810d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d370081bc7cb45ecab9c396c9bd130ff, user_id=eda88161285a4fb5b4b72cd6f68bf4ec, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:42.903142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978cfdd9-234d-47d9-860b-b272589b810d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:43.314619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 985/1943] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:42 2022] PUT /identity/v3/projects/d370081bc7cb45ecab9c396c9bd130ff/users/eda88161285a4fb5b4b72cd6f68bf4ec/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:43.328300 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d044d-a1ec-46f7-91ae-26be0dd14a93 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:43.328582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d044d-a1ec-46f7-91ae-26be0dd14a93 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:43.328848 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d044d-a1ec-46f7-91ae-26be0dd14a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:43.360037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-245d044d-a1ec-46f7-91ae-26be0dd14a93 None None] MFA Rules not processed for user `eda88161285a4fb5b4b72cd6f68bf4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:43.361314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-245d044d-a1ec-46f7-91ae-26be0dd14a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:43.467147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 959/1944] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:43.471996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cfdd9-234d-47d9-860b-b272589b810d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:43.480700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:43.566975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cfdd9-234d-47d9-860b-b272589b810d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:43.578074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea1096-62b8-42c0-830e-493b5d91ff4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:43.579417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea1096-62b8-42c0-830e-493b5d91ff4f admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:43.579924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea1096-62b8-42c0-830e-493b5d91ff4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:43.580297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea1096-62b8-42c0-830e-493b5d91ff4f admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:43.581111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea1096-62b8-42c0-830e-493b5d91ff4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:43.584581 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea1096-62b8-42c0-830e-493b5d91ff4f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:43.726688 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 986/1945] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:43 2022] POST /identity/v3/projects => generated 367 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:27:43.727169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d044d-a1ec-46f7-91ae-26be0dd14a93 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:43.733836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d044d-a1ec-46f7-91ae-26be0dd14a93 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:43.781441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:43.872251 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ab15d-2635-4fe4-bf36-a5c4c49a4e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:43.872887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ab15d-2635-4fe4-bf36-a5c4c49a4e17 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:43.873157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ab15d-2635-4fe4-bf36-a5c4c49a4e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:43.873612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ab15d-2635-4fe4-bf36-a5c4c49a4e17 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:43.876458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ab15d-2635-4fe4-bf36-a5c4c49a4e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:43.881798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ab15d-2635-4fe4-bf36-a5c4c49a4e17 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:44.110715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 960/1946] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:43 2022] POST /identity/v3/users => generated 290 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:44.114871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea1096-62b8-42c0-830e-493b5d91ff4f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:44.132572 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea1096-62b8-42c0-830e-493b5d91ff4f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:44.162759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9555cc-2517-478c-8b3a-2505abf5532b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:44.162759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9555cc-2517-478c-8b3a-2505abf5532b admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:44.162759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9555cc-2517-478c-8b3a-2505abf5532b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:44.162759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9555cc-2517-478c-8b3a-2505abf5532b admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:44.164013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9555cc-2517-478c-8b3a-2505abf5532b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:44.164840 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9555cc-2517-478c-8b3a-2505abf5532b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:44.209826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ab15d-2635-4fe4-bf36-a5c4c49a4e17 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:44.210762 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 987/1947] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:44.233203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ab15d-2635-4fe4-bf36-a5c4c49a4e17 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:44.243091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5c300-86f8-4dbe-b0db-0e7bfcaecaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:44.244738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5c300-86f8-4dbe-b0db-0e7bfcaecaa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:44.245523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5c300-86f8-4dbe-b0db-0e7bfcaecaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:44.246250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5c300-86f8-4dbe-b0db-0e7bfcaecaa8 admin admin] PATH_INFO: `/v3/projects/4df0d86513cc4ff5a6b0e2969fb0e4cb/users/3f874a7a14114b2d980944862cc486a7/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:44.351316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5c300-86f8-4dbe-b0db-0e7bfcaecaa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df0d86513cc4ff5a6b0e2969fb0e4cb, user_id=3f874a7a14114b2d980944862cc486a7, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:44.359821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5c300-86f8-4dbe-b0db-0e7bfcaecaa8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:44.604522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 961/1948] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:44 2022] PUT /identity/v3/projects/4df0d86513cc4ff5a6b0e2969fb0e4cb/users/3f874a7a14114b2d980944862cc486a7/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:44.610117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9555cc-2517-478c-8b3a-2505abf5532b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:44.634947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9555cc-2517-478c-8b3a-2505abf5532b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:44.842384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce07540-fdb8-4cad-83b0-471239a653f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:44.843599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce07540-fdb8-4cad-83b0-471239a653f2 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:44.843884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce07540-fdb8-4cad-83b0-471239a653f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:44.844158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce07540-fdb8-4cad-83b0-471239a653f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:44.845082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce07540-fdb8-4cad-83b0-471239a653f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:44.846977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce07540-fdb8-4cad-83b0-471239a653f2 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:44.878263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 988/1949] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:27:44.886621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5c300-86f8-4dbe-b0db-0e7bfcaecaa8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:44.926836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5c300-86f8-4dbe-b0db-0e7bfcaecaa8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:44.961407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f4acd-556f-4410-8be4-59ff61721326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:44.964897 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f4acd-556f-4410-8be4-59ff61721326 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:44.966993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f4acd-556f-4410-8be4-59ff61721326 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:44.968944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f4acd-556f-4410-8be4-59ff61721326 admin admin] PATH_INFO: `/v3/projects/4df0d86513cc4ff5a6b0e2969fb0e4cb/users/3f874a7a14114b2d980944862cc486a7/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:44.979267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f4acd-556f-4410-8be4-59ff61721326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df0d86513cc4ff5a6b0e2969fb0e4cb, user_id=3f874a7a14114b2d980944862cc486a7, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:44.987783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f4acd-556f-4410-8be4-59ff61721326 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:45.059507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 962/1950] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:44 2022] PUT /identity/v3/projects/4df0d86513cc4ff5a6b0e2969fb0e4cb/users/3f874a7a14114b2d980944862cc486a7/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:45.082195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1d21a-2054-4dcd-ace2-289ce5b74478 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:45.088138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1d21a-2054-4dcd-ace2-289ce5b74478 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:45.088138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1d21a-2054-4dcd-ace2-289ce5b74478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:45.145441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-10a1d21a-2054-4dcd-ace2-289ce5b74478 None None] MFA Rules not processed for user `3f874a7a14114b2d980944862cc486a7`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:45.150905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-10a1d21a-2054-4dcd-ace2-289ce5b74478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:45.254031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 989/1951] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:45.277427 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f4acd-556f-4410-8be4-59ff61721326 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:45.282340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-745f4acd-556f-4410-8be4-59ff61721326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:45.416529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f4acd-556f-4410-8be4-59ff61721326 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:45.438025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eb146-72e7-4a45-9e8f-835796229607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:45.441999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eb146-72e7-4a45-9e8f-835796229607 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:45.445230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eb146-72e7-4a45-9e8f-835796229607 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:45.445230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eb146-72e7-4a45-9e8f-835796229607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:45.468898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3eb146-72e7-4a45-9e8f-835796229607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:45.473500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3eb146-72e7-4a45-9e8f-835796229607 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:45.507319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 963/1952] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:45.760324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1d21a-2054-4dcd-ace2-289ce5b74478 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:45.791484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1d21a-2054-4dcd-ace2-289ce5b74478 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:45.795866 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954022c6-8d5f-48e2-ab5f-54e0cf390a3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:45.805874 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954022c6-8d5f-48e2-ab5f-54e0cf390a3c service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:45.806524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954022c6-8d5f-48e2-ab5f-54e0cf390a3c service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:45.809502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954022c6-8d5f-48e2-ab5f-54e0cf390a3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:45.926510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954022c6-8d5f-48e2-ab5f-54e0cf390a3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:45.930119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954022c6-8d5f-48e2-ab5f-54e0cf390a3c service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:45.952166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 990/1953] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:45.963090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:47.544063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eb146-72e7-4a45-9e8f-835796229607 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:47.549361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3f3eb146-72e7-4a45-9e8f-835796229607 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:47.653239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:47.698790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eb146-72e7-4a45-9e8f-835796229607 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:47.704405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:47.705101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:47.705374 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:47.705651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] PATH_INFO: `/v3/users/f371764cbe374a5fafb0d0b8e94d405c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:47.706940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f371764cbe374a5fafb0d0b8e94d405c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:47.709565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:48.012720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f371764cbe374a5fafb0d0b8e94d405c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:48.014792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f371764cbe374a5fafb0d0b8e94d405c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:48.016518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f371764cbe374a5fafb0d0b8e94d405c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:48.038034 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f371764cbe374a5fafb0d0b8e94d405c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:48.070840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f371764cbe374a5fafb0d0b8e94d405c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:48.186015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 964/1954] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:47 2022] DELETE /identity/v3/users/f371764cbe374a5fafb0d0b8e94d405c => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:48.193200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fdfea-cf66-425a-87e8-c842c1e8a27a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:48.194336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fdfea-cf66-425a-87e8-c842c1e8a27a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:48.194976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fdfea-cf66-425a-87e8-c842c1e8a27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:48.238122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-be7fdfea-cf66-425a-87e8-c842c1e8a27a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:48.241969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-be7fdfea-cf66-425a-87e8-c842c1e8a27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:48.381767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 991/1955] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:48.393877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:48.395481 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:48.486820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee3937-63c6-42be-b6fd-714acd971375 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:48.502074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779e5a9-a815-42c7-ada6-caf9e7f5d8b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:48.504094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779e5a9-a815-42c7-ada6-caf9e7f5d8b7 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:48.504387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779e5a9-a815-42c7-ada6-caf9e7f5d8b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:48.505270 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779e5a9-a815-42c7-ada6-caf9e7f5d8b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:48.545815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779e5a9-a815-42c7-ada6-caf9e7f5d8b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:48.563606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779e5a9-a815-42c7-ada6-caf9e7f5d8b7 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:48.630051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 965/1956] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:49.512749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fdfea-cf66-425a-87e8-c842c1e8a27a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:49.515098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-be7fdfea-cf66-425a-87e8-c842c1e8a27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:49.540382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fdfea-cf66-425a-87e8-c842c1e8a27a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:49.546545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:49.547625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:49.548603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:49.549317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] PATH_INFO: `/v3/projects/8f2c7f3618004988ae2b62a1100a9f17` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:49.551267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f2c7f3618004988ae2b62a1100a9f17)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:49.561770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:49.595359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779e5a9-a815-42c7-ada6-caf9e7f5d8b7 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:49.599054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e779e5a9-a815-42c7-ada6-caf9e7f5d8b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:49.647445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779e5a9-a815-42c7-ada6-caf9e7f5d8b7 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:49.664792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:49.666110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:49.666957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:49.667446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] PATH_INFO: `/v3/users/0da9b6f63ffe4dcd89c7ce1f0918c506` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:49.669095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da9b6f63ffe4dcd89c7ce1f0918c506)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:49.671634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:49.752121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f2c7f3618004988ae2b62a1100a9f17', 'request_id': 'req-911a0a11-6e27-4a3d-ae7e-cb459886d29a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:49.781214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f2c7f3618004988ae2b62a1100a9f17', 'request_id': 'req-911a0a11-6e27-4a3d-ae7e-cb459886d29a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:49.796433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f2c7f3618004988ae2b62a1100a9f17', 'request_id': 'req-911a0a11-6e27-4a3d-ae7e-cb459886d29a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:49.910871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] The token cache is being invalidate because project 8f2c7f3618004988ae2b62a1100a9f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:49.911501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:49.912530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:49.915186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 992/1957] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:49 2022] DELETE /identity/v3/projects/8f2c7f3618004988ae2b62a1100a9f17 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:49.919564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:49.922080 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:49.962908 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da9b6f63ffe4dcd89c7ce1f0918c506'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:49.963909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da9b6f63ffe4dcd89c7ce1f0918c506'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:49.964924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da9b6f63ffe4dcd89c7ce1f0918c506'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:49.973652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:49.976367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da9b6f63ffe4dcd89c7ce1f0918c506'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:49.993296 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-26239d7c-5bd9-4189-86a0-a5425686092a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da9b6f63ffe4dcd89c7ce1f0918c506'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:50.007148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:50.062140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a0a11-6e27-4a3d-ae7e-cb459886d29a admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:50.068426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:50.069331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:50.070553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:50.070947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] PATH_INFO: `/v3/users/89484b16dc0949fcb8868494b401b812` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:50.072619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] RBAC: Authorizing `identity:delete_user(user_id=89484b16dc0949fcb8868494b401b812)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:50.074761 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:50.130140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 966/1958] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:49 2022] DELETE /identity/v3/users/0da9b6f63ffe4dcd89c7ce1f0918c506 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:50.137414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90303ee0-214a-4cdf-9b29-dcc72e80e61e None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:50.137835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90303ee0-214a-4cdf-9b29-dcc72e80e61e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:50.138248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90303ee0-214a-4cdf-9b29-dcc72e80e61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:50.167496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-90303ee0-214a-4cdf-9b29-dcc72e80e61e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:50.168721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-90303ee0-214a-4cdf-9b29-dcc72e80e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:50.247270 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 967/1959] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:50.260795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90303ee0-214a-4cdf-9b29-dcc72e80e61e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:50.263789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-90303ee0-214a-4cdf-9b29-dcc72e80e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:50.340618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90303ee0-214a-4cdf-9b29-dcc72e80e61e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:50.345140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94493ff3-7961-4319-ad3f-0fc59a04b3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:50.345784 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94493ff3-7961-4319-ad3f-0fc59a04b3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:50.346117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94493ff3-7961-4319-ad3f-0fc59a04b3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:50.346395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94493ff3-7961-4319-ad3f-0fc59a04b3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:50.373878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94493ff3-7961-4319-ad3f-0fc59a04b3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:50.375458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94493ff3-7961-4319-ad3f-0fc59a04b3f4 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:50.379520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89484b16dc0949fcb8868494b401b812'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:50.380960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89484b16dc0949fcb8868494b401b812'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:50.381457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89484b16dc0949fcb8868494b401b812'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:50.397078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 968/1960] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:27:50.470915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89484b16dc0949fcb8868494b401b812'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:50.481507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89484b16dc0949fcb8868494b401b812'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:50.527010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 993/1961] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:49 2022] DELETE /identity/v3/users/89484b16dc0949fcb8868494b401b812 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:50.927286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94493ff3-7961-4319-ad3f-0fc59a04b3f4 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:50.930225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-94493ff3-7961-4319-ad3f-0fc59a04b3f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:50.961509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94493ff3-7961-4319-ad3f-0fc59a04b3f4 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:51.048028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:51.049006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:51.049363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:51.049682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] PATH_INFO: `/v3/projects/646c9aa0fe82428ca4d65dd4e132ffde` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:51.052420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] RBAC: Authorizing `identity:delete_project(project_id=646c9aa0fe82428ca4d65dd4e132ffde)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:51.057032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:51.090150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:51.222998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:51.226830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:51.236402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646c9aa0fe82428ca4d65dd4e132ffde', 'request_id': 'req-b30ed769-d71a-40f3-b514-21729da750df'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.245604 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646c9aa0fe82428ca4d65dd4e132ffde', 'request_id': 'req-b30ed769-d71a-40f3-b514-21729da750df'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.277001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00322559-752e-4548-a5e1-f9ffc83f7aae admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:51.427924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:51.429472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:51.429472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:51.431663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646c9aa0fe82428ca4d65dd4e132ffde', 'request_id': 'req-b30ed769-d71a-40f3-b514-21729da750df'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.434890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] PATH_INFO: `/v3/projects/51d58cdb5d59490195696f6ea2a882de` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:51.437034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51d58cdb5d59490195696f6ea2a882de)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:51.455892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] The token cache is being invalidate because project 646c9aa0fe82428ca4d65dd4e132ffde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:51.457050 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.457244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:51.479700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.483012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 969/1962] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:50 2022] DELETE /identity/v3/projects/646c9aa0fe82428ca4d65dd4e132ffde => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:51.488892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:51.493919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:51.529214 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ed769-d71a-40f3-b514-21729da750df admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:51.551584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:51.551584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:51.551584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:51.551584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] PATH_INFO: `/v3/users/d4651259a8314ef79ea36f602f3fc5af` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:51.552432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4651259a8314ef79ea36f602f3fc5af)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:51.555070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:51.682009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:51.689774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51d58cdb5d59490195696f6ea2a882de', 'request_id': 'req-a49aff4d-21a7-401e-8657-7525f7f54ef3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.697681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51d58cdb5d59490195696f6ea2a882de', 'request_id': 'req-a49aff4d-21a7-401e-8657-7525f7f54ef3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.705450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51d58cdb5d59490195696f6ea2a882de', 'request_id': 'req-a49aff4d-21a7-401e-8657-7525f7f54ef3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.813524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] The token cache is being invalidate because project 51d58cdb5d59490195696f6ea2a882de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:51.813827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.814675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.816378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 994/1963] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:51 2022] DELETE /identity/v3/projects/51d58cdb5d59490195696f6ea2a882de => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:51.824001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:51.825573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:51.884357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4651259a8314ef79ea36f602f3fc5af'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.885858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4651259a8314ef79ea36f602f3fc5af'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.887540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4651259a8314ef79ea36f602f3fc5af'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.921878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4651259a8314ef79ea36f602f3fc5af'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.935146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aff4d-21a7-401e-8657-7525f7f54ef3 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:51.951672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:51.952663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:51.953081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:51.953355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] PATH_INFO: `/v3/users/bf81e3b8dffa4cffb178aa4b5fee2a04` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:51.955076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4651259a8314ef79ea36f602f3fc5af'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:51.964624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf81e3b8dffa4cffb178aa4b5fee2a04)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:51.966912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:52.049144 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 970/1964] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:51 2022] DELETE /identity/v3/users/d4651259a8314ef79ea36f602f3fc5af => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:52.296078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf81e3b8dffa4cffb178aa4b5fee2a04'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:52.297445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf81e3b8dffa4cffb178aa4b5fee2a04'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:52.298739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf81e3b8dffa4cffb178aa4b5fee2a04'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:52.431361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf81e3b8dffa4cffb178aa4b5fee2a04'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:52.447797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf81e3b8dffa4cffb178aa4b5fee2a04'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:52.474552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 995/1965] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:51 2022] DELETE /identity/v3/users/bf81e3b8dffa4cffb178aa4b5fee2a04 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:52.730852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:52.733249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:52.871441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:52.873451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:52.991268 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf918-1d56-420f-b5e0-b6e080fd8b44 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:53.062601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:53.075270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754f5d6-1fc8-4661-8b44-1fd8dfc8cb39 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:53.088695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:53.091716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:53.092391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:53.093451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] PATH_INFO: `/v3/users/eda88161285a4fb5b4b72cd6f68bf4ec` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:53.096558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda88161285a4fb5b4b72cd6f68bf4ec)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:53.102003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:53.157105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:53.196538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:53.198995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:53.200143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:53.200994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] PATH_INFO: `/v3/projects/87ae5cd8fb1e4a7dbc2e3bb4705924e3` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:53.203987 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ae5cd8fb1e4a7dbc2e3bb4705924e3)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:53.208882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:53.434008 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda88161285a4fb5b4b72cd6f68bf4ec'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.436189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda88161285a4fb5b4b72cd6f68bf4ec'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.439930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda88161285a4fb5b4b72cd6f68bf4ec'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.441943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ae5cd8fb1e4a7dbc2e3bb4705924e3', 'request_id': 'req-d522b750-b242-47d3-9e96-92ae83ab2980'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.450995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ae5cd8fb1e4a7dbc2e3bb4705924e3', 'request_id': 'req-d522b750-b242-47d3-9e96-92ae83ab2980'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.574355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda88161285a4fb5b4b72cd6f68bf4ec'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.584681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda88161285a4fb5b4b72cd6f68bf4ec'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.611363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 996/1966] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:52 2022] DELETE /identity/v3/users/eda88161285a4fb5b4b72cd6f68bf4ec => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:53.611363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:53.611363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:53.710913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ae5cd8fb1e4a7dbc2e3bb4705924e3', 'request_id': 'req-d522b750-b242-47d3-9e96-92ae83ab2980'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.722067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce328be8-ba5c-4db6-a243-8de932e4e7cc admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:53.732666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:53.735782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:53.736260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:53.736660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] PATH_INFO: `/v3/projects/c041a33b44a54c7ab108fa84e48a65e4` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:53.740489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c041a33b44a54c7ab108fa84e48a65e4)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:53.742731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:53.743791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] The token cache is being invalidate because project 87ae5cd8fb1e4a7dbc2e3bb4705924e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:53.744570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.747479 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d522b750-b242-47d3-9e96-92ae83ab2980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.755318 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 971/1967] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:52 2022] DELETE /identity/v3/projects/87ae5cd8fb1e4a7dbc2e3bb4705924e3 => generated 0 bytes in 1024 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:53.761384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8327a-a2a3-4155-ac8d-7127986fe510 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:53.762453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8327a-a2a3-4155-ac8d-7127986fe510 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:53.766588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8327a-a2a3-4155-ac8d-7127986fe510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:53.814713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-51b8327a-a2a3-4155-ac8d-7127986fe510 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:53.816290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-51b8327a-a2a3-4155-ac8d-7127986fe510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:53.830662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 972/1968] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:53.830662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662bffa-b732-4b5f-b96e-ab1febb709d3 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:53.830662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662bffa-b732-4b5f-b96e-ab1febb709d3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:53.830662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662bffa-b732-4b5f-b96e-ab1febb709d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:53.859871 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c662bffa-b732-4b5f-b96e-ab1febb709d3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:53.865464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c662bffa-b732-4b5f-b96e-ab1febb709d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:53.875378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 973/1969] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:53.885955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb48291-39ab-4bb2-b687-b2a3ab9eccaf None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:53.886395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb48291-39ab-4bb2-b687-b2a3ab9eccaf None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:53.886987 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb48291-39ab-4bb2-b687-b2a3ab9eccaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:53.934001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-fbb48291-39ab-4bb2-b687-b2a3ab9eccaf None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:53.939540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fbb48291-39ab-4bb2-b687-b2a3ab9eccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:53.960050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c041a33b44a54c7ab108fa84e48a65e4', 'request_id': 'req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.988746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c041a33b44a54c7ab108fa84e48a65e4', 'request_id': 'req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:53.998402 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c041a33b44a54c7ab108fa84e48a65e4', 'request_id': 'req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:54.029790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 974/1970] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:54.031726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb48291-39ab-4bb2-b687-b2a3ab9eccaf None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.033639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fbb48291-39ab-4bb2-b687-b2a3ab9eccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:54.124758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] The token cache is being invalidate because project c041a33b44a54c7ab108fa84e48a65e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:54.135226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:54.136835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:54.141180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 997/1971] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:53 2022] DELETE /identity/v3/projects/c041a33b44a54c7ab108fa84e48a65e4 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:54.143002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.149400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:54.182725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39345a8-f1a4-4da6-9f6f-ec84ad228cfb admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:54.193518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb48291-39ab-4bb2-b687-b2a3ab9eccaf None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:54.204249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10724083-086f-46d0-9c98-a7981843d52d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:54.205141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10724083-086f-46d0-9c98-a7981843d52d service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.205753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10724083-086f-46d0-9c98-a7981843d52d service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.206273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10724083-086f-46d0-9c98-a7981843d52d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.207490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088e878-f428-481c-8bf8-17d9f196ce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:54.208728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088e878-f428-481c-8bf8-17d9f196ce63 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.209070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088e878-f428-481c-8bf8-17d9f196ce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.212024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088e878-f428-481c-8bf8-17d9f196ce63 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.212125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-10724083-086f-46d0-9c98-a7981843d52d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:54.212227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088e878-f428-481c-8bf8-17d9f196ce63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:27:54.212227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088e878-f428-481c-8bf8-17d9f196ce63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:54.212712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088e878-f428-481c-8bf8-17d9f196ce63 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:54.231749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 998/1972] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:27:54.233647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c951f0c-98a5-4cbd-8ed0-da80ead1c638 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.234150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c951f0c-98a5-4cbd-8ed0-da80ead1c638 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.234668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c951f0c-98a5-4cbd-8ed0-da80ead1c638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.237749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10724083-086f-46d0-9c98-a7981843d52d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:54.241257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10724083-086f-46d0-9c98-a7981843d52d service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:54.263667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2c951f0c-98a5-4cbd-8ed0-da80ead1c638 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:54.264652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 975/1973] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:54.265972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2c951f0c-98a5-4cbd-8ed0-da80ead1c638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:54.267410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10724083-086f-46d0-9c98-a7981843d52d service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.278664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 999/1974] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:54.289809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb55e6b-ea7e-402d-854c-a787725594a7 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.290335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb55e6b-ea7e-402d-854c-a787725594a7 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.290824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb55e6b-ea7e-402d-854c-a787725594a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.291712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10724083-086f-46d0-9c98-a7981843d52d service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:54.302274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230e71f-bf9c-4e23-be7a-e027f942e341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:54.321387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230e71f-bf9c-4e23-be7a-e027f942e341 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.322358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230e71f-bf9c-4e23-be7a-e027f942e341 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.323178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230e71f-bf9c-4e23-be7a-e027f942e341 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.324747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230e71f-bf9c-4e23-be7a-e027f942e341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:54.328658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6bb55e6b-ea7e-402d-854c-a787725594a7 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:54.330293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6bb55e6b-ea7e-402d-854c-a787725594a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:54.336277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230e71f-bf9c-4e23-be7a-e027f942e341 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:54.341883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1000/1975] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:54.351003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb55e6b-ea7e-402d-854c-a787725594a7 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.383479 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb55e6b-ea7e-402d-854c-a787725594a7 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:54.391134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b3660-12ef-436a-93e9-3752e30dd68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:54.392646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b3660-12ef-436a-93e9-3752e30dd68d admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.392646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b3660-12ef-436a-93e9-3752e30dd68d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.392646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b3660-12ef-436a-93e9-3752e30dd68d admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.393430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b3660-12ef-436a-93e9-3752e30dd68d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:27:54.393678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b3660-12ef-436a-93e9-3752e30dd68d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:54.396029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b3660-12ef-436a-93e9-3752e30dd68d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:54.420060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1001/1976] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:27:54.421091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b3660-12ef-436a-93e9-3752e30dd68d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.441258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b3660-12ef-436a-93e9-3752e30dd68d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:54.451962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd29bf-1721-4b06-b8f9-285bccfbe200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:54.453051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd29bf-1721-4b06-b8f9-285bccfbe200 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.454199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd29bf-1721-4b06-b8f9-285bccfbe200 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.454865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd29bf-1721-4b06-b8f9-285bccfbe200 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.456017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd29bf-1721-4b06-b8f9-285bccfbe200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:54.460848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd29bf-1721-4b06-b8f9-285bccfbe200 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:54.526741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 976/1977] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:54 2022] POST /identity/v3/projects => generated 377 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:54.534204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230e71f-bf9c-4e23-be7a-e027f942e341 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.612292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1002/1978] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:27:54 2022] POST /identity/v3/projects => generated 391 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:54.614192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd29bf-1721-4b06-b8f9-285bccfbe200 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.638305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230e71f-bf9c-4e23-be7a-e027f942e341 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:54.729350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c23f17-2c27-4e73-a195-102ac8800214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:54.730132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c23f17-2c27-4e73-a195-102ac8800214 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.730432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c23f17-2c27-4e73-a195-102ac8800214 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.730752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c23f17-2c27-4e73-a195-102ac8800214 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.731391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c23f17-2c27-4e73-a195-102ac8800214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:54.733575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c23f17-2c27-4e73-a195-102ac8800214 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:54.771027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd29bf-1721-4b06-b8f9-285bccfbe200 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:54.778649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4479d7-b596-4a41-9761-e9931894911f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:54.779314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4479d7-b596-4a41-9761-e9931894911f admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.779597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4479d7-b596-4a41-9761-e9931894911f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.779871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4479d7-b596-4a41-9761-e9931894911f admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.780543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4479d7-b596-4a41-9761-e9931894911f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:54.782810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4479d7-b596-4a41-9761-e9931894911f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:54.843489 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 977/1979] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:54 2022] POST /identity/v3/users => generated 289 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:27:54.848086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c23f17-2c27-4e73-a195-102ac8800214 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.860954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c23f17-2c27-4e73-a195-102ac8800214 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:54.888268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1003/1980] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:27:54 2022] POST /identity/v3/users => generated 296 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:27:54.894744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4479d7-b596-4a41-9761-e9931894911f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.900437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f453c-9124-4b80-9b99-f315235aa35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:54.901685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f453c-9124-4b80-9b99-f315235aa35c admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.902316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f453c-9124-4b80-9b99-f315235aa35c admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.902953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f453c-9124-4b80-9b99-f315235aa35c admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.905343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f453c-9124-4b80-9b99-f315235aa35c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:54.909658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f453c-9124-4b80-9b99-f315235aa35c admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:54.912657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4479d7-b596-4a41-9761-e9931894911f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:54.926735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bb43e-29bc-434b-b0a6-d2c94e96bc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:54.929202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bb43e-29bc-434b-b0a6-d2c94e96bc98 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.929202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bb43e-29bc-434b-b0a6-d2c94e96bc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.929202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bb43e-29bc-434b-b0a6-d2c94e96bc98 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.931095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bb43e-29bc-434b-b0a6-d2c94e96bc98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:54.932886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bb43e-29bc-434b-b0a6-d2c94e96bc98 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:54.941245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 978/1981] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:27:54.946760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f453c-9124-4b80-9b99-f315235aa35c admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.966851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1004/1982] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:54.968821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bb43e-29bc-434b-b0a6-d2c94e96bc98 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:54.970542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f453c-9124-4b80-9b99-f315235aa35c admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:54.977833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17109ce4-28d0-4790-9dfb-23e8e005ea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:54.978626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109ce4-28d0-4790-9dfb-23e8e005ea43 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:54.979028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109ce4-28d0-4790-9dfb-23e8e005ea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:54.979479 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109ce4-28d0-4790-9dfb-23e8e005ea43 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:54.985236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17109ce4-28d0-4790-9dfb-23e8e005ea43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:54.985236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17109ce4-28d0-4790-9dfb-23e8e005ea43 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:54.993637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bb43e-29bc-434b-b0a6-d2c94e96bc98 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:55.002077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 979/1983] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:55.004361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17109ce4-28d0-4790-9dfb-23e8e005ea43 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:55.024338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17109ce4-28d0-4790-9dfb-23e8e005ea43 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:55.028219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab8063-4382-4abf-a53f-aae82028407f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:55.034125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab8063-4382-4abf-a53f-aae82028407f admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:55.043862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcea851-a52c-4a26-a568-f5473834ba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:55.048218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcea851-a52c-4a26-a568-f5473834ba99 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:55.050283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcea851-a52c-4a26-a568-f5473834ba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:55.050788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcea851-a52c-4a26-a568-f5473834ba99 admin admin] PATH_INFO: `/v3/projects/4de02c4fd1fb48d589ab31a1eadc2cd6/users/710633b814b14e23a13a142aa12fc797/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:55.056157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab8063-4382-4abf-a53f-aae82028407f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:55.056801 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab8063-4382-4abf-a53f-aae82028407f admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:55.066679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab8063-4382-4abf-a53f-aae82028407f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:55.077410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab8063-4382-4abf-a53f-aae82028407f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:55.107951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcea851-a52c-4a26-a568-f5473834ba99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de02c4fd1fb48d589ab31a1eadc2cd6, user_id=710633b814b14e23a13a142aa12fc797, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:55.119211 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcea851-a52c-4a26-a568-f5473834ba99 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:55.145698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1005/1984] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:27:55.149544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab8063-4382-4abf-a53f-aae82028407f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:55.171283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab8063-4382-4abf-a53f-aae82028407f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:55.171283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186be9c5-cdea-4aad-88dd-78dd441419a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:55.171283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186be9c5-cdea-4aad-88dd-78dd441419a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:55.171283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186be9c5-cdea-4aad-88dd-78dd441419a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:55.171860 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186be9c5-cdea-4aad-88dd-78dd441419a3 admin admin] PATH_INFO: `/v3/projects/c6751708050444a4a04f19a2072a4c64/users/601be948ace540caac3e54af70092b3e/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:55.200756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186be9c5-cdea-4aad-88dd-78dd441419a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6751708050444a4a04f19a2072a4c64, user_id=601be948ace540caac3e54af70092b3e, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:55.200756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186be9c5-cdea-4aad-88dd-78dd441419a3 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:55.241805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 980/1985] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:55 2022] PUT /identity/v3/projects/4de02c4fd1fb48d589ab31a1eadc2cd6/users/710633b814b14e23a13a142aa12fc797/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:55.264731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc96898-9ecb-48fa-b922-258d75d4407f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:55.265136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc96898-9ecb-48fa-b922-258d75d4407f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:55.265501 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc96898-9ecb-48fa-b922-258d75d4407f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:55.326525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1006/1986] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:27:55 2022] PUT /identity/v3/projects/c6751708050444a4a04f19a2072a4c64/users/601be948ace540caac3e54af70092b3e/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:27:55.331408 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-cfc96898-9ecb-48fa-b922-258d75d4407f None None] MFA Rules not processed for user `710633b814b14e23a13a142aa12fc797`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:55.333704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cfc96898-9ecb-48fa-b922-258d75d4407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:55.345084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f957def-3d11-4e6e-a1e2-c105b097d42d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:55.345689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f957def-3d11-4e6e-a1e2-c105b097d42d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:55.346585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f957def-3d11-4e6e-a1e2-c105b097d42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:55.355239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:55.425417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:55.438587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-9f957def-3d11-4e6e-a1e2-c105b097d42d None None] MFA Rules not processed for user `601be948ace540caac3e54af70092b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:27:55.441167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9f957def-3d11-4e6e-a1e2-c105b097d42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:55.555432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 981/1987] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:27:55.560961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc96898-9ecb-48fa-b922-258d75d4407f None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:55.563747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cfc96898-9ecb-48fa-b922-258d75d4407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:55.691612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1007/1988] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:27:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:27:55.709851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f957def-3d11-4e6e-a1e2-c105b097d42d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:55.713580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9f957def-3d11-4e6e-a1e2-c105b097d42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:55.797188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:55.814270 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc96898-9ecb-48fa-b922-258d75d4407f None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:55.820748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e789c4b-b7e2-4461-a633-c8e24e05e729 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea4c2870e2754f3188228fc441950df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:55.822837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e789c4b-b7e2-4461-a633-c8e24e05e729 service glance] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:55.822837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e789c4b-b7e2-4461-a633-c8e24e05e729 service glance] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:55.822837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e789c4b-b7e2-4461-a633-c8e24e05e729 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:55.840945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e789c4b-b7e2-4461-a633-c8e24e05e729 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:55.843717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e789c4b-b7e2-4461-a633-c8e24e05e729 service glance] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:55.864883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:55.866542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 982/1989] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:55.874429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f957def-3d11-4e6e-a1e2-c105b097d42d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:55.881128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269458b7-ca48-4313-8dea-05365cc85594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:55.882063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269458b7-ca48-4313-8dea-05365cc85594 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:55.882441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269458b7-ca48-4313-8dea-05365cc85594 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:55.882847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269458b7-ca48-4313-8dea-05365cc85594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:55.897956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269458b7-ca48-4313-8dea-05365cc85594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:55.901525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269458b7-ca48-4313-8dea-05365cc85594 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:55.927958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1008/1990] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:27:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:27:56.418360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e789c4b-b7e2-4461-a633-c8e24e05e729 service glance] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:56.421153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5e789c4b-b7e2-4461-a633-c8e24e05e729 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:27:56.665507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e789c4b-b7e2-4461-a633-c8e24e05e729 service glance] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:56.687701 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:56.688617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:56.689075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:56.689527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] PATH_INFO: `/v3/projects/d370081bc7cb45ecab9c396c9bd130ff` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:56.699418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d370081bc7cb45ecab9c396c9bd130ff)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:56.700771 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:56.902446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:56.947262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd370081bc7cb45ecab9c396c9bd130ff', 'request_id': 'req-5f8275b0-9cd9-41a9-b64c-e25c49df472a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:56.977685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd370081bc7cb45ecab9c396c9bd130ff', 'request_id': 'req-5f8275b0-9cd9-41a9-b64c-e25c49df472a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:57.117446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd370081bc7cb45ecab9c396c9bd130ff', 'request_id': 'req-5f8275b0-9cd9-41a9-b64c-e25c49df472a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:57.227403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] The token cache is being invalidate because project d370081bc7cb45ecab9c396c9bd130ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:27:57.228249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:57.232763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:57.246859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 983/1991] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:27:56 2022] DELETE /identity/v3/projects/d370081bc7cb45ecab9c396c9bd130ff => generated 0 bytes in 825 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:27:57.255531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269458b7-ca48-4313-8dea-05365cc85594 service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:27:57.297751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269458b7-ca48-4313-8dea-05365cc85594 service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:27:57.532002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:27:57.533775 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:27:57.534379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:27:57.535002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] PATH_INFO: `/v3/users/3f874a7a14114b2d980944862cc486a7` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:27:57.538243 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f874a7a14114b2d980944862cc486a7)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:27:57.542694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:27:57.869174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f874a7a14114b2d980944862cc486a7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:57.870962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f874a7a14114b2d980944862cc486a7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:57.872364 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f874a7a14114b2d980944862cc486a7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:57.996494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f874a7a14114b2d980944862cc486a7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:58.005124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f874a7a14114b2d980944862cc486a7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:27:58.041420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:27:58.065582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1009/1992] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:27:57 2022] DELETE /identity/v3/users/3f874a7a14114b2d980944862cc486a7 => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:00.090137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:00.093546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:00.127955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8275b0-9cd9-41a9-b64c-e25c49df472a admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:00.245292 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:00.245964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:00.246238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:00.246551 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] PATH_INFO: `/v3/users/f3e34404618b4f32b5021aa5a2fec7f1` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:00.248113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e34404618b4f32b5021aa5a2fec7f1)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:00.250252 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:00.400498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:00.408246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:00.435086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8ba77-c20a-4eb2-bb76-db3764fbc24e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:00.441244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:00.442227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:00.442706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:00.443115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] PATH_INFO: `/v3/projects/4df0d86513cc4ff5a6b0e2969fb0e4cb` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:00.444617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4df0d86513cc4ff5a6b0e2969fb0e4cb)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:00.447137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:00.523078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e34404618b4f32b5021aa5a2fec7f1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:00.523862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e34404618b4f32b5021aa5a2fec7f1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:00.524574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e34404618b4f32b5021aa5a2fec7f1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:00.533623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e34404618b4f32b5021aa5a2fec7f1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:00.572363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b6a99b1d-b8a1-4e1d-a8f4-5bb67b128685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e34404618b4f32b5021aa5a2fec7f1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:00.599728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4df0d86513cc4ff5a6b0e2969fb0e4cb', 'request_id': 'req-fbe9a312-9ca0-436a-b49e-e191eb0be101'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:00.608566 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4df0d86513cc4ff5a6b0e2969fb0e4cb', 'request_id': 'req-fbe9a312-9ca0-436a-b49e-e191eb0be101'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:00.623192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4df0d86513cc4ff5a6b0e2969fb0e4cb', 'request_id': 'req-fbe9a312-9ca0-436a-b49e-e191eb0be101'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:00.678711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 984/1993] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:00 2022] DELETE /identity/v3/users/f3e34404618b4f32b5021aa5a2fec7f1 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:00.760756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] The token cache is being invalidate because project 4df0d86513cc4ff5a6b0e2969fb0e4cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:00.761232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:00.762171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fbe9a312-9ca0-436a-b49e-e191eb0be101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:00.764341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1010/1994] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:00 2022] DELETE /identity/v3/projects/4df0d86513cc4ff5a6b0e2969fb0e4cb => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:00.773748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a6b2f-9168-40a9-88b1-d92a449d968c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:00.774254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a6b2f-9168-40a9-88b1-d92a449d968c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:00.778698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a6b2f-9168-40a9-88b1-d92a449d968c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:00.805196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-e40a6b2f-9168-40a9-88b1-d92a449d968c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:00.806579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e40a6b2f-9168-40a9-88b1-d92a449d968c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:00.848015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:00.886263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 985/1995] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:00.896874 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa3915-651d-406c-8865-8415cc1cc9a9 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:00.897301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa3915-651d-406c-8865-8415cc1cc9a9 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:00.897812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa3915-651d-406c-8865-8415cc1cc9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:00.941059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2caa3915-651d-406c-8865-8415cc1cc9a9 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:00.944156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2caa3915-651d-406c-8865-8415cc1cc9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:00.967906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1011/1996] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:00.968548 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a6b2f-9168-40a9-88b1-d92a449d968c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:00.981811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a6b2f-9168-40a9-88b1-d92a449d968c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:00.990182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630136f-2c79-4473-bee6-f3bbe40edb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:00.991161 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630136f-2c79-4473-bee6-f3bbe40edb1b admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:00.991482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630136f-2c79-4473-bee6-f3bbe40edb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:00.991909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630136f-2c79-4473-bee6-f3bbe40edb1b admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:00.992919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630136f-2c79-4473-bee6-f3bbe40edb1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:00.993335 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630136f-2c79-4473-bee6-f3bbe40edb1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:00.996181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630136f-2c79-4473-bee6-f3bbe40edb1b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:01.021164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 986/1997] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:28:01.022452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa3915-651d-406c-8865-8415cc1cc9a9 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:01.033414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa3915-651d-406c-8865-8415cc1cc9a9 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:01.039449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b979f1e-af72-45f1-a023-d6d6336eb248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:01.040277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b979f1e-af72-45f1-a023-d6d6336eb248 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:01.040668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b979f1e-af72-45f1-a023-d6d6336eb248 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:01.041035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b979f1e-af72-45f1-a023-d6d6336eb248 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:01.041914 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b979f1e-af72-45f1-a023-d6d6336eb248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:01.044153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b979f1e-af72-45f1-a023-d6d6336eb248 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:01.071770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630136f-2c79-4473-bee6-f3bbe40edb1b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:01.074834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9630136f-2c79-4473-bee6-f3bbe40edb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:01.095203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630136f-2c79-4473-bee6-f3bbe40edb1b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:01.102142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:01.102802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:01.103244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:01.103704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] PATH_INFO: `/v3/projects/efb9b57f8f324d4a92d4a311d13009a4` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:01.105253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb9b57f8f324d4a92d4a311d13009a4)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:01.108584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:01.174551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1012/1998] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:01 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:01.180000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b979f1e-af72-45f1-a023-d6d6336eb248 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:01.253337 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb9b57f8f324d4a92d4a311d13009a4', 'request_id': 'req-dfc8c077-467d-4301-9f3c-a3d41cb6b438'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:01.268216 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb9b57f8f324d4a92d4a311d13009a4', 'request_id': 'req-dfc8c077-467d-4301-9f3c-a3d41cb6b438'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:01.306092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b979f1e-af72-45f1-a023-d6d6336eb248 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:01.381141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb9b57f8f324d4a92d4a311d13009a4', 'request_id': 'req-dfc8c077-467d-4301-9f3c-a3d41cb6b438'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:01.411825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] The token cache is being invalidate because project efb9b57f8f324d4a92d4a311d13009a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:01.412324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:01.413554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:01.416358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:01.421281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 987/1999] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:01 2022] DELETE /identity/v3/projects/efb9b57f8f324d4a92d4a311d13009a4 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:01.423132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:01.425433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:01.428013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b301e-1122-4904-b8d7-16043c09781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:01.428648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b301e-1122-4904-b8d7-16043c09781b admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:01.429028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b301e-1122-4904-b8d7-16043c09781b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:01.429269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b301e-1122-4904-b8d7-16043c09781b admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:01.429924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b301e-1122-4904-b8d7-16043c09781b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:01.432203 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b301e-1122-4904-b8d7-16043c09781b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:01.455755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8c077-467d-4301-9f3c-a3d41cb6b438 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:01.465061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:01.466135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:01.466773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:01.467257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] PATH_INFO: `/v3/users/9ad75420642d433bbf7b491302a3276c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:01.469911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ad75420642d433bbf7b491302a3276c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:01.473975 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:01.547622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1013/2000] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:01 2022] POST /identity/v3/users => generated 291 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:01.549374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b301e-1122-4904-b8d7-16043c09781b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:01.552763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7b5b301e-1122-4904-b8d7-16043c09781b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:01.565480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b301e-1122-4904-b8d7-16043c09781b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:01.571686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1863c2-4e10-4f80-9107-5440341e98c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:01.572324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1863c2-4e10-4f80-9107-5440341e98c8 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:01.572604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1863c2-4e10-4f80-9107-5440341e98c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:01.572876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1863c2-4e10-4f80-9107-5440341e98c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:01.573536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1863c2-4e10-4f80-9107-5440341e98c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:01.575499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1863c2-4e10-4f80-9107-5440341e98c8 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:01.588693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1014/2001] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:01.593369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1863c2-4e10-4f80-9107-5440341e98c8 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:01.604513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1863c2-4e10-4f80-9107-5440341e98c8 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:01.609736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcafd67-d544-40e3-a108-20a636c41184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:01.610367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcafd67-d544-40e3-a108-20a636c41184 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:01.610679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcafd67-d544-40e3-a108-20a636c41184 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:01.610954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcafd67-d544-40e3-a108-20a636c41184 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:01.611608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcafd67-d544-40e3-a108-20a636c41184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:01.613284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcafd67-d544-40e3-a108-20a636c41184 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:01.631266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1015/2002] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:01.633432 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcafd67-d544-40e3-a108-20a636c41184 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:01.646151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcafd67-d544-40e3-a108-20a636c41184 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:01.651899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c824b-72a7-4b66-9137-0c17b603bb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:01.652779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c824b-72a7-4b66-9137-0c17b603bb5a admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:01.652935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c824b-72a7-4b66-9137-0c17b603bb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:01.653125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c824b-72a7-4b66-9137-0c17b603bb5a admin admin] PATH_INFO: `/v3/projects/917f7c77af7b42c9bda701b163137e18/users/ea56232675364a708bb01ee083d59add/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:01.680215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c824b-72a7-4b66-9137-0c17b603bb5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=917f7c77af7b42c9bda701b163137e18, user_id=ea56232675364a708bb01ee083d59add, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:01.684417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c824b-72a7-4b66-9137-0c17b603bb5a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:01.765856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ad75420642d433bbf7b491302a3276c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:01.767001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ad75420642d433bbf7b491302a3276c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:01.767684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ad75420642d433bbf7b491302a3276c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:01.779167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ad75420642d433bbf7b491302a3276c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:01.800295 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ad75420642d433bbf7b491302a3276c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:01.806570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1016/2003] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:01 2022] PUT /identity/v3/projects/917f7c77af7b42c9bda701b163137e18/users/ea56232675364a708bb01ee083d59add/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:01.818469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013eaf-f061-4cdb-b34d-0303a05377db None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:01.818931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013eaf-f061-4cdb-b34d-0303a05377db None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:01.819346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013eaf-f061-4cdb-b34d-0303a05377db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:01.851696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-07013eaf-f061-4cdb-b34d-0303a05377db None None] MFA Rules not processed for user `ea56232675364a708bb01ee083d59add`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:01.852951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-07013eaf-f061-4cdb-b34d-0303a05377db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:01.893011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 988/2004] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:01 2022] DELETE /identity/v3/users/9ad75420642d433bbf7b491302a3276c => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:01.935006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1017/2005] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:01.943343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:01.945009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:01.980941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:02.054229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa181cd-62e8-4706-b821-083113572930 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:02.059593 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2438e71-b5a9-4a1f-8fa8-81f4b283c912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:02.060255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2438e71-b5a9-4a1f-8fa8-81f4b283c912 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:02.060490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2438e71-b5a9-4a1f-8fa8-81f4b283c912 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:02.060789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2438e71-b5a9-4a1f-8fa8-81f4b283c912 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:02.061445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2438e71-b5a9-4a1f-8fa8-81f4b283c912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:02.063511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2438e71-b5a9-4a1f-8fa8-81f4b283c912 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:02.246537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 989/2006] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:01 2022] POST /identity/v3/projects => generated 379 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:02.248043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013eaf-f061-4cdb-b34d-0303a05377db None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:02.283296 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2438e71-b5a9-4a1f-8fa8-81f4b283c912 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:02.286359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e2438e71-b5a9-4a1f-8fa8-81f4b283c912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:02.378839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013eaf-f061-4cdb-b34d-0303a05377db None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:02.422088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2438e71-b5a9-4a1f-8fa8-81f4b283c912 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:02.428169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:02.428854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:02.429143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:02.429407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] PATH_INFO: `/v3/projects/2bb3baf24cdc43a48e2549ba3429ac27` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:02.430737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb3baf24cdc43a48e2549ba3429ac27)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:02.432690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:02.444396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:02.491918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa705b37-a600-4cce-9feb-5a3a20bc168d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:02.492553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa705b37-a600-4cce-9feb-5a3a20bc168d admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:02.492823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa705b37-a600-4cce-9feb-5a3a20bc168d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:02.493091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa705b37-a600-4cce-9feb-5a3a20bc168d admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:02.493781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa705b37-a600-4cce-9feb-5a3a20bc168d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:02.495833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa705b37-a600-4cce-9feb-5a3a20bc168d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:02.594546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb3baf24cdc43a48e2549ba3429ac27', 'request_id': 'req-76b4a9b0-95fe-419d-987b-fd32b00ee382'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:02.603475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb3baf24cdc43a48e2549ba3429ac27', 'request_id': 'req-76b4a9b0-95fe-419d-987b-fd32b00ee382'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:02.612430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1018/2007] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:02 2022] POST /identity/v3/users => generated 296 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:02.616662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa705b37-a600-4cce-9feb-5a3a20bc168d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:02.632867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa705b37-a600-4cce-9feb-5a3a20bc168d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:02.700528 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb3baf24cdc43a48e2549ba3429ac27', 'request_id': 'req-76b4a9b0-95fe-419d-987b-fd32b00ee382'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:02.733436 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] The token cache is being invalidate because project 2bb3baf24cdc43a48e2549ba3429ac27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:02.734358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:02.735525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-76b4a9b0-95fe-419d-987b-fd32b00ee382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:02.738513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 990/2008] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:02 2022] DELETE /identity/v3/projects/2bb3baf24cdc43a48e2549ba3429ac27 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:02.748618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120ed81-18b4-4ceb-a6f1-804f9907b97a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:02.748889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f98935-ea20-4b92-95e8-832a952bc6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:02.749460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120ed81-18b4-4ceb-a6f1-804f9907b97a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:02.749758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f98935-ea20-4b92-95e8-832a952bc6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:02.750182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120ed81-18b4-4ceb-a6f1-804f9907b97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:02.750342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f98935-ea20-4b92-95e8-832a952bc6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:02.750679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f98935-ea20-4b92-95e8-832a952bc6b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:02.751807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f98935-ea20-4b92-95e8-832a952bc6b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:02.759172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f98935-ea20-4b92-95e8-832a952bc6b5 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:02.775863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1019/2009] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:02.780872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f98935-ea20-4b92-95e8-832a952bc6b5 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:02.784037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-57f98935-ea20-4b92-95e8-832a952bc6b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:02.790557 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-6120ed81-18b4-4ceb-a6f1-804f9907b97a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:02.793190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-6120ed81-18b4-4ceb-a6f1-804f9907b97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:02.812519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 991/2010] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:02.823793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f98935-ea20-4b92-95e8-832a952bc6b5 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:02.825800 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb0f29-08bd-4592-9f27-197b6836b50b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:02.826262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb0f29-08bd-4592-9f27-197b6836b50b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:02.827074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb0f29-08bd-4592-9f27-197b6836b50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:02.836520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3d33a-3a67-4422-8b86-ff02d49a55f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:02.837756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3d33a-3a67-4422-8b86-ff02d49a55f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:02.838305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3d33a-3a67-4422-8b86-ff02d49a55f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:02.839310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3d33a-3a67-4422-8b86-ff02d49a55f0 admin admin] PATH_INFO: `/v3/projects/ab3b8a6092f14864b55e2f2c0597a686/users/ad54c3095c95436e93c1d4d7a7bc7e24/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:02.862973 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3d33a-3a67-4422-8b86-ff02d49a55f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab3b8a6092f14864b55e2f2c0597a686, user_id=ad54c3095c95436e93c1d4d7a7bc7e24, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:02.866610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d8eb0f29-08bd-4592-9f27-197b6836b50b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:02.866871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3d33a-3a67-4422-8b86-ff02d49a55f0 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:02.868761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d8eb0f29-08bd-4592-9f27-197b6836b50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:02.885533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 992/2011] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:02.893240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb0f29-08bd-4592-9f27-197b6836b50b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:02.912088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb0f29-08bd-4592-9f27-197b6836b50b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:02.924838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4592a94-c3bc-44d6-855b-eeab95380f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:02.925951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4592a94-c3bc-44d6-855b-eeab95380f2d admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:02.927668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4592a94-c3bc-44d6-855b-eeab95380f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:02.928149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4592a94-c3bc-44d6-855b-eeab95380f2d admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:02.929259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4592a94-c3bc-44d6-855b-eeab95380f2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:02.929794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4592a94-c3bc-44d6-855b-eeab95380f2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:02.932944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4592a94-c3bc-44d6-855b-eeab95380f2d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:02.953481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1020/2012] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:02 2022] PUT /identity/v3/projects/ab3b8a6092f14864b55e2f2c0597a686/users/ad54c3095c95436e93c1d4d7a7bc7e24/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:02.961355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 993/2013] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:28:02.961560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3d33a-3a67-4422-8b86-ff02d49a55f0 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:02.963504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4592a94-c3bc-44d6-855b-eeab95380f2d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:02.986665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4592a94-c3bc-44d6-855b-eeab95380f2d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:02.993324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:03.067395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3d33a-3a67-4422-8b86-ff02d49a55f0 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:03.072820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dfc0a-1235-4926-80c8-a975ad285240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:03.073464 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dfc0a-1235-4926-80c8-a975ad285240 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:03.073748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dfc0a-1235-4926-80c8-a975ad285240 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:03.074025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dfc0a-1235-4926-80c8-a975ad285240 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:03.074736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442dfc0a-1235-4926-80c8-a975ad285240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:03.080661 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442dfc0a-1235-4926-80c8-a975ad285240 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:03.098686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1021/2014] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:03.098955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70a5a9-f34a-4504-a95a-1f01a96898e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:03.099138 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70a5a9-f34a-4504-a95a-1f01a96898e8 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:03.099289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70a5a9-f34a-4504-a95a-1f01a96898e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:03.099566 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70a5a9-f34a-4504-a95a-1f01a96898e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:03.100449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70a5a9-f34a-4504-a95a-1f01a96898e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:03.102734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70a5a9-f34a-4504-a95a-1f01a96898e8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:03.103402 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dfc0a-1235-4926-80c8-a975ad285240 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:03.121533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dfc0a-1235-4926-80c8-a975ad285240 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:03.129523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc26a70-3685-402d-85dc-b84b392e37ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:03.134813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc26a70-3685-402d-85dc-b84b392e37ef admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:03.134813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc26a70-3685-402d-85dc-b84b392e37ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:03.134813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc26a70-3685-402d-85dc-b84b392e37ef admin admin] PATH_INFO: `/v3/projects/ab3b8a6092f14864b55e2f2c0597a686/users/ad54c3095c95436e93c1d4d7a7bc7e24/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:03.135379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc26a70-3685-402d-85dc-b84b392e37ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab3b8a6092f14864b55e2f2c0597a686, user_id=ad54c3095c95436e93c1d4d7a7bc7e24, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:03.140921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc26a70-3685-402d-85dc-b84b392e37ef admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:03.161588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1022/2015] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:03 2022] PUT /identity/v3/projects/ab3b8a6092f14864b55e2f2c0597a686/users/ad54c3095c95436e93c1d4d7a7bc7e24/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:03.181897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e51551-366e-4a5a-a7de-7d3240856002 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:03.182360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e51551-366e-4a5a-a7de-7d3240856002 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:03.182873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e51551-366e-4a5a-a7de-7d3240856002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:03.238451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 994/2016] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:02 2022] POST /identity/v3/projects => generated 371 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:03.239778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-84e51551-366e-4a5a-a7de-7d3240856002 None None] MFA Rules not processed for user `ad54c3095c95436e93c1d4d7a7bc7e24`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:03.240359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70a5a9-f34a-4504-a95a-1f01a96898e8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:03.241949 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-84e51551-366e-4a5a-a7de-7d3240856002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:03.252430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:03.340114 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70a5a9-f34a-4504-a95a-1f01a96898e8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:03.346183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d5fe-689b-4f37-a9be-93d15783bd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:03.347120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d5fe-689b-4f37-a9be-93d15783bd9c admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:03.347523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d5fe-689b-4f37-a9be-93d15783bd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:03.347926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d5fe-689b-4f37-a9be-93d15783bd9c admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:03.349393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721d5fe-689b-4f37-a9be-93d15783bd9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:03.351474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721d5fe-689b-4f37-a9be-93d15783bd9c admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:03.357061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1023/2017] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:03.372017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e51551-366e-4a5a-a7de-7d3240856002 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:03.373600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-84e51551-366e-4a5a-a7de-7d3240856002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:03.447012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e51551-366e-4a5a-a7de-7d3240856002 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:03.451899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee3dc3-46e4-4e6f-b763-4854ef82b295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:03.453273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee3dc3-46e4-4e6f-b763-4854ef82b295 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:03.453273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee3dc3-46e4-4e6f-b763-4854ef82b295 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:03.453273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee3dc3-46e4-4e6f-b763-4854ef82b295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:03.467609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee3dc3-46e4-4e6f-b763-4854ef82b295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:03.469493 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee3dc3-46e4-4e6f-b763-4854ef82b295 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:03.485901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1024/2018] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:28:03.488455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 995/2019] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:03 2022] POST /identity/v3/users => generated 286 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:03.493584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee3dc3-46e4-4e6f-b763-4854ef82b295 service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:03.511715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee3dc3-46e4-4e6f-b763-4854ef82b295 service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:03.519751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e65550-34d1-472b-8f59-b69acc4d998b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:03.521290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65550-34d1-472b-8f59-b69acc4d998b admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:03.521745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65550-34d1-472b-8f59-b69acc4d998b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:03.522218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65550-34d1-472b-8f59-b69acc4d998b admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:03.523418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65550-34d1-472b-8f59-b69acc4d998b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:03.526531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65550-34d1-472b-8f59-b69acc4d998b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:03.545508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1025/2020] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:03.548065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d5fe-689b-4f37-a9be-93d15783bd9c admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:03.569955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d5fe-689b-4f37-a9be-93d15783bd9c admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:03.581310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b3cf4-331c-404a-84d5-c73e01cd12d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:03.582561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b3cf4-331c-404a-84d5-c73e01cd12d3 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:03.583465 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b3cf4-331c-404a-84d5-c73e01cd12d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:03.584364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b3cf4-331c-404a-84d5-c73e01cd12d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:03.584957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b3cf4-331c-404a-84d5-c73e01cd12d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:03.587420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b3cf4-331c-404a-84d5-c73e01cd12d3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:03.613413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 996/2021] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:03.619207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e65550-34d1-472b-8f59-b69acc4d998b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:03.636182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e65550-34d1-472b-8f59-b69acc4d998b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:03.641632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed30db0-d2e8-47de-bbf2-cb604847f5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:03.642321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed30db0-d2e8-47de-bbf2-cb604847f5dd admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:03.642650 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed30db0-d2e8-47de-bbf2-cb604847f5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:03.642924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed30db0-d2e8-47de-bbf2-cb604847f5dd admin admin] PATH_INFO: `/v3/projects/84ff1f60d005454a90e647f1ace0cf43/users/03a65de47d5d41ab858191a32058bcdb/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:03.664095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed30db0-d2e8-47de-bbf2-cb604847f5dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ff1f60d005454a90e647f1ace0cf43, user_id=03a65de47d5d41ab858191a32058bcdb, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:03.669259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed30db0-d2e8-47de-bbf2-cb604847f5dd admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:03.779110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1026/2022] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:03 2022] PUT /identity/v3/projects/84ff1f60d005454a90e647f1ace0cf43/users/03a65de47d5d41ab858191a32058bcdb/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:03.797927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd91c3-970d-43f7-98fb-23bbb029060f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:03.798786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd91c3-970d-43f7-98fb-23bbb029060f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:03.799374 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd91c3-970d-43f7-98fb-23bbb029060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:03.862399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3cbd91c3-970d-43f7-98fb-23bbb029060f None None] MFA Rules not processed for user `03a65de47d5d41ab858191a32058bcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:03.865984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3cbd91c3-970d-43f7-98fb-23bbb029060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:04.007906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:04.019547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 997/2023] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:04.031419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed30db0-d2e8-47de-bbf2-cb604847f5dd admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:04.034780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2ed30db0-d2e8-47de-bbf2-cb604847f5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:04.121822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed30db0-d2e8-47de-bbf2-cb604847f5dd admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:04.126474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790557b8-b1ab-41e1-8fb3-f7d4f4b75925 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea4c2870e2754f3188228fc441950df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:04.127696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790557b8-b1ab-41e1-8fb3-f7d4f4b75925 service glance] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:04.128081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790557b8-b1ab-41e1-8fb3-f7d4f4b75925 service glance] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:04.128790 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790557b8-b1ab-41e1-8fb3-f7d4f4b75925 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:04.140394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790557b8-b1ab-41e1-8fb3-f7d4f4b75925 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:04.142116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790557b8-b1ab-41e1-8fb3-f7d4f4b75925 service glance] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:04.154468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1027/2024] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:28:04.765614 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd91c3-970d-43f7-98fb-23bbb029060f None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:04.840411 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd91c3-970d-43f7-98fb-23bbb029060f None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:04.845546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00950f-820c-4584-886c-06911e6e3f8a tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad54c3095c95436e93c1d4d7a7bc7e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3b8a6092f14864b55e2f2c0597a686', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:04.846342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00950f-820c-4584-886c-06911e6e3f8a tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:04.846710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00950f-820c-4584-886c-06911e6e3f8a tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:04.847031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00950f-820c-4584-886c-06911e6e3f8a tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:04.847889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00950f-820c-4584-886c-06911e6e3f8a tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:04.849047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00950f-820c-4584-886c-06911e6e3f8a tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:04.851239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00950f-820c-4584-886c-06911e6e3f8a tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:04.866642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 998/2025] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:28:04.871830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790557b8-b1ab-41e1-8fb3-f7d4f4b75925 service glance] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:04.886618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790557b8-b1ab-41e1-8fb3-f7d4f4b75925 service glance] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:04.892669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759431ff-e751-4bd8-9836-59e8bc08df99 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad54c3095c95436e93c1d4d7a7bc7e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3b8a6092f14864b55e2f2c0597a686', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:04.893520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759431ff-e751-4bd8-9836-59e8bc08df99 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:04.893883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759431ff-e751-4bd8-9836-59e8bc08df99 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:04.894241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759431ff-e751-4bd8-9836-59e8bc08df99 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:04.895555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759431ff-e751-4bd8-9836-59e8bc08df99 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:04.897985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759431ff-e751-4bd8-9836-59e8bc08df99 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:04.911322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:05.019530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1028/2026] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:04 2022] POST /identity/v3/projects => generated 355 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:05.024347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00950f-820c-4584-886c-06911e6e3f8a tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:05.037323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00950f-820c-4584-886c-06911e6e3f8a tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:05.195003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df544e2-20df-4c29-aa96-d5c414355377 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad54c3095c95436e93c1d4d7a7bc7e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3b8a6092f14864b55e2f2c0597a686', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:05.210588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df544e2-20df-4c29-aa96-d5c414355377 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:05.211213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df544e2-20df-4c29-aa96-d5c414355377 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:05.212628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df544e2-20df-4c29-aa96-d5c414355377 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:05.213031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df544e2-20df-4c29-aa96-d5c414355377 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:05.213519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df544e2-20df-4c29-aa96-d5c414355377 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:05.217704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df544e2-20df-4c29-aa96-d5c414355377 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:05.238544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 999/2027] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:28:05.313315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759431ff-e751-4bd8-9836-59e8bc08df99 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:05.335506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759431ff-e751-4bd8-9836-59e8bc08df99 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:05.341472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad54c3095c95436e93c1d4d7a7bc7e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3b8a6092f14864b55e2f2c0597a686', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:05.342341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:05.342744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:05.343337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] PATH_INFO: `/v3/projects/06d20edfde2d4345a971b5fd11514987` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:05.344677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: Authorizing `identity:delete_project(project_id=06d20edfde2d4345a971b5fd11514987)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:05.347142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:05.513041 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d20edfde2d4345a971b5fd11514987', 'request_id': 'req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:05.523894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d20edfde2d4345a971b5fd11514987', 'request_id': 'req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:05.530518 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d20edfde2d4345a971b5fd11514987', 'request_id': 'req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:05.654041 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] The token cache is being invalidate because project 06d20edfde2d4345a971b5fd11514987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:05.654395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:05.655287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6a4a1c7c-fc28-4ca6-9ed5-ba1a9e125382 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:05.656972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1029/2028] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:05 2022] DELETE /identity/v3/projects/06d20edfde2d4345a971b5fd11514987 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:06.444856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df544e2-20df-4c29-aa96-d5c414355377 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:06.447011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2df544e2-20df-4c29-aa96-d5c414355377 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:06.532090 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df544e2-20df-4c29-aa96-d5c414355377 tempest-VolumeQuotaClassesTest-180836618 tempest-VolumeQuotaClassesTest-180836618-project-admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:06.588115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:06.621917 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:06.623109 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:06.623580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:06.624082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] PATH_INFO: `/v3/users/ea56232675364a708bb01ee083d59add` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:06.625834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea56232675364a708bb01ee083d59add)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:06.628385 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:06.901627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea56232675364a708bb01ee083d59add'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:06.902400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea56232675364a708bb01ee083d59add'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:06.903227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea56232675364a708bb01ee083d59add'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:06.913102 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea56232675364a708bb01ee083d59add'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:06.928890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea56232675364a708bb01ee083d59add'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:07.003530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1000/2029] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:06 2022] DELETE /identity/v3/users/ea56232675364a708bb01ee083d59add => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:07.008698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6afbf7-47ce-4837-b8b7-07e30f9b9437 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:07.009497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6afbf7-47ce-4837-b8b7-07e30f9b9437 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:07.009497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6afbf7-47ce-4837-b8b7-07e30f9b9437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:07.031190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-dc6afbf7-47ce-4837-b8b7-07e30f9b9437 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:07.032336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dc6afbf7-47ce-4837-b8b7-07e30f9b9437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:07.114862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1030/2030] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:07.126944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:07.131973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:07.227926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8653abd-d05c-43d3-b757-0cd1a2364d03 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:07.233390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c25c2-c694-4420-a1f4-106c99a2c4c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:07.234297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c25c2-c694-4420-a1f4-106c99a2c4c5 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:07.234859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c25c2-c694-4420-a1f4-106c99a2c4c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:07.240963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c25c2-c694-4420-a1f4-106c99a2c4c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:07.262025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c25c2-c694-4420-a1f4-106c99a2c4c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:07.265542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c25c2-c694-4420-a1f4-106c99a2c4c5 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:07.288970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1001/2031] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:28:07.835648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6afbf7-47ce-4837-b8b7-07e30f9b9437 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:07.838240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dc6afbf7-47ce-4837-b8b7-07e30f9b9437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:07.866469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6afbf7-47ce-4837-b8b7-07e30f9b9437 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:07.874276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:07.875199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:07.875669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:07.876104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] PATH_INFO: `/v3/projects/917f7c77af7b42c9bda701b163137e18` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:07.877723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=917f7c77af7b42c9bda701b163137e18)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:07.880053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:08.024574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917f7c77af7b42c9bda701b163137e18', 'request_id': 'req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:08.032601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917f7c77af7b42c9bda701b163137e18', 'request_id': 'req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:08.040787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917f7c77af7b42c9bda701b163137e18', 'request_id': 'req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:08.106966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:08.116197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] The token cache is being invalidate because project 917f7c77af7b42c9bda701b163137e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:08.116549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:08.117363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:08.119175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1031/2032] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:07 2022] DELETE /identity/v3/projects/917f7c77af7b42c9bda701b163137e18 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:08.123786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c25c2-c694-4420-a1f4-106c99a2c4c5 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:08.138854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c25c2-c694-4420-a1f4-106c99a2c4c5 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:08.234107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:08.242737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:08.243450 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:08.243711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:08.244014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] PATH_INFO: `/v3/users/ad54c3095c95436e93c1d4d7a7bc7e24` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:08.245268 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad54c3095c95436e93c1d4d7a7bc7e24)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:08.247344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:08.516318 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad54c3095c95436e93c1d4d7a7bc7e24'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:08.519642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad54c3095c95436e93c1d4d7a7bc7e24'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:08.521042 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad54c3095c95436e93c1d4d7a7bc7e24'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:08.531098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad54c3095c95436e93c1d4d7a7bc7e24'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:08.548363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f222d3bc-4899-43fe-9c27-d2b16ca62952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad54c3095c95436e93c1d4d7a7bc7e24'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:08.621412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1002/2033] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:08 2022] DELETE /identity/v3/users/ad54c3095c95436e93c1d4d7a7bc7e24 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:09.133197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:09.135470 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:09.156102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28ad9a-2c39-48d3-898b-0bbd41866ee4 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:09.268376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:09.269042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:09.269322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:09.269608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] PATH_INFO: `/v3/projects/ab3b8a6092f14864b55e2f2c0597a686` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:09.271500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab3b8a6092f14864b55e2f2c0597a686)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:09.273791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:09.431724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab3b8a6092f14864b55e2f2c0597a686', 'request_id': 'req-1536610a-74ef-46f7-9edb-617bd1dc680e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:09.441135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab3b8a6092f14864b55e2f2c0597a686', 'request_id': 'req-1536610a-74ef-46f7-9edb-617bd1dc680e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:09.451217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab3b8a6092f14864b55e2f2c0597a686', 'request_id': 'req-1536610a-74ef-46f7-9edb-617bd1dc680e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:09.535421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] The token cache is being invalidate because project ab3b8a6092f14864b55e2f2c0597a686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:09.536542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:09.537627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1536610a-74ef-46f7-9edb-617bd1dc680e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:09.539637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1032/2034] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:09 2022] DELETE /identity/v3/projects/ab3b8a6092f14864b55e2f2c0597a686 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:09.546412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a2196-f61a-49f6-b757-f585ae1a957e None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:09.546952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a2196-f61a-49f6-b757-f585ae1a957e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:09.547354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a2196-f61a-49f6-b757-f585ae1a957e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:09.583181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d19a2196-f61a-49f6-b757-f585ae1a957e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:09.585462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d19a2196-f61a-49f6-b757-f585ae1a957e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:09.682744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1003/2035] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:09.690952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664042cd-6a52-4c14-b9c5-12a8c0d8fb63 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:09.691230 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664042cd-6a52-4c14-b9c5-12a8c0d8fb63 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:09.691509 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664042cd-6a52-4c14-b9c5-12a8c0d8fb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:09.715657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-664042cd-6a52-4c14-b9c5-12a8c0d8fb63 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:09.716889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-664042cd-6a52-4c14-b9c5-12a8c0d8fb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:09.730537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1033/2036] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:09.733122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a2196-f61a-49f6-b757-f585ae1a957e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:09.752899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a2196-f61a-49f6-b757-f585ae1a957e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:09.761443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80edf0-bea3-4958-8e8d-0912750cca13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:09.762536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80edf0-bea3-4958-8e8d-0912750cca13 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:09.763047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80edf0-bea3-4958-8e8d-0912750cca13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:09.763530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80edf0-bea3-4958-8e8d-0912750cca13 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:09.764992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80edf0-bea3-4958-8e8d-0912750cca13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:09.765532 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80edf0-bea3-4958-8e8d-0912750cca13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:09.769048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80edf0-bea3-4958-8e8d-0912750cca13 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:09.793128 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1004/2037] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:28:09.797998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664042cd-6a52-4c14-b9c5-12a8c0d8fb63 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:09.810082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664042cd-6a52-4c14-b9c5-12a8c0d8fb63 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:09.815499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8f94b-73f8-4d08-a41e-558aab3dfa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:09.816168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8f94b-73f8-4d08-a41e-558aab3dfa5c admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:09.816482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8f94b-73f8-4d08-a41e-558aab3dfa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:09.816791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8f94b-73f8-4d08-a41e-558aab3dfa5c admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:09.818404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8f94b-73f8-4d08-a41e-558aab3dfa5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:09.820748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8f94b-73f8-4d08-a41e-558aab3dfa5c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:09.909962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1034/2038] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:09 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:09.915657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80edf0-bea3-4958-8e8d-0912750cca13 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:09.928400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80edf0-bea3-4958-8e8d-0912750cca13 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:10.005747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e13ddc-ae85-40aa-b44d-04dcf54bb8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:10.006510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e13ddc-ae85-40aa-b44d-04dcf54bb8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:10.006802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e13ddc-ae85-40aa-b44d-04dcf54bb8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:10.007074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e13ddc-ae85-40aa-b44d-04dcf54bb8d3 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:10.007724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e13ddc-ae85-40aa-b44d-04dcf54bb8d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:10.009870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e13ddc-ae85-40aa-b44d-04dcf54bb8d3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:10.021629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:10.150912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1005/2039] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:09 2022] POST /identity/v3/users => generated 294 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:10.157603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8f94b-73f8-4d08-a41e-558aab3dfa5c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:10.175802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8f94b-73f8-4d08-a41e-558aab3dfa5c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:10.184460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fb4d3-4c5f-4861-a39a-df82f6f859fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:10.185401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fb4d3-4c5f-4861-a39a-df82f6f859fb admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:10.185954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fb4d3-4c5f-4861-a39a-df82f6f859fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:10.186425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fb4d3-4c5f-4861-a39a-df82f6f859fb admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:10.187663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fb4d3-4c5f-4861-a39a-df82f6f859fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:10.190767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fb4d3-4c5f-4861-a39a-df82f6f859fb admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:10.205892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1035/2040] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:10.213108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e13ddc-ae85-40aa-b44d-04dcf54bb8d3 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:10.234255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e13ddc-ae85-40aa-b44d-04dcf54bb8d3 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:10.240560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0fc90-d1f2-4ea0-8ec7-a763c08ccdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:10.241195 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0fc90-d1f2-4ea0-8ec7-a763c08ccdbb admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:10.241490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0fc90-d1f2-4ea0-8ec7-a763c08ccdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:10.241741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0fc90-d1f2-4ea0-8ec7-a763c08ccdbb admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:10.242401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0fc90-d1f2-4ea0-8ec7-a763c08ccdbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:10.244094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0fc90-d1f2-4ea0-8ec7-a763c08ccdbb admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:10.258067 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1006/2041] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:10.263291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fb4d3-4c5f-4861-a39a-df82f6f859fb admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:10.277390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fb4d3-4c5f-4861-a39a-df82f6f859fb admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:10.284613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62092057-8a1b-4376-9d25-c9fa9a4c4c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:10.285948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62092057-8a1b-4376-9d25-c9fa9a4c4c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:10.287327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62092057-8a1b-4376-9d25-c9fa9a4c4c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:10.287327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62092057-8a1b-4376-9d25-c9fa9a4c4c9d admin admin] PATH_INFO: `/v3/projects/c6837919a3f7439785e65ee360bbfb24/users/4e18ecf2cbd8424d80c023bb2ed50a41/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:10.305646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62092057-8a1b-4376-9d25-c9fa9a4c4c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6837919a3f7439785e65ee360bbfb24, user_id=4e18ecf2cbd8424d80c023bb2ed50a41, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:10.308705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62092057-8a1b-4376-9d25-c9fa9a4c4c9d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:10.430820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1036/2042] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:10 2022] PUT /identity/v3/projects/c6837919a3f7439785e65ee360bbfb24/users/4e18ecf2cbd8424d80c023bb2ed50a41/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:10.459825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcf894-b6dc-41c5-946a-5de946e3e133 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:10.461440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcf894-b6dc-41c5-946a-5de946e3e133 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:10.461887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcf894-b6dc-41c5-946a-5de946e3e133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:10.520915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-befcf894-b6dc-41c5-946a-5de946e3e133 None None] MFA Rules not processed for user `4e18ecf2cbd8424d80c023bb2ed50a41`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:10.523208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-befcf894-b6dc-41c5-946a-5de946e3e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:10.646376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1007/2043] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:10.651465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62092057-8a1b-4376-9d25-c9fa9a4c4c9d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:10.700228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:10.746106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62092057-8a1b-4376-9d25-c9fa9a4c4c9d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:10.753004 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922c7e3-a34e-45a7-bbd6-839eb7adce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:10.753854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922c7e3-a34e-45a7-bbd6-839eb7adce64 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:10.754918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922c7e3-a34e-45a7-bbd6-839eb7adce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:10.756234 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922c7e3-a34e-45a7-bbd6-839eb7adce64 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:10.757114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922c7e3-a34e-45a7-bbd6-839eb7adce64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:10.760355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922c7e3-a34e-45a7-bbd6-839eb7adce64 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:10.901456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1037/2044] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:10 2022] POST /identity/v3/projects => generated 389 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:10.906135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcf894-b6dc-41c5-946a-5de946e3e133 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:10.920186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcf894-b6dc-41c5-946a-5de946e3e133 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:11.010551 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8cbed-27a3-40eb-ba20-8375a87c6258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:11.011079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8cbed-27a3-40eb-ba20-8375a87c6258 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:11.011358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8cbed-27a3-40eb-ba20-8375a87c6258 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:11.011677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8cbed-27a3-40eb-ba20-8375a87c6258 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:11.012341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8cbed-27a3-40eb-ba20-8375a87c6258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:11.014649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8cbed-27a3-40eb-ba20-8375a87c6258 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:11.107151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1008/2045] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:10 2022] POST /identity/v3/users => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:11.112591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922c7e3-a34e-45a7-bbd6-839eb7adce64 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:11.126178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922c7e3-a34e-45a7-bbd6-839eb7adce64 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:11.132535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350b9d7-fa47-4029-b072-b84bb71286bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:11.133249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350b9d7-fa47-4029-b072-b84bb71286bf admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:11.134821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350b9d7-fa47-4029-b072-b84bb71286bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:11.134821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350b9d7-fa47-4029-b072-b84bb71286bf admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:11.134821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350b9d7-fa47-4029-b072-b84bb71286bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:11.136466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350b9d7-fa47-4029-b072-b84bb71286bf admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:11.154494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1038/2046] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:11.161159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8cbed-27a3-40eb-ba20-8375a87c6258 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:11.172849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8cbed-27a3-40eb-ba20-8375a87c6258 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:11.181551 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45bda8-7814-4bda-90c8-096e021aa92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:11.182256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45bda8-7814-4bda-90c8-096e021aa92a admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:11.182582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45bda8-7814-4bda-90c8-096e021aa92a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:11.182860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45bda8-7814-4bda-90c8-096e021aa92a admin admin] PATH_INFO: `/v3/projects/f03708a315eb41ee906b1e9d105e5f5a/users/c8be46f058714afab5d47b7bb700c1ff/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:11.201511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45bda8-7814-4bda-90c8-096e021aa92a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03708a315eb41ee906b1e9d105e5f5a, user_id=c8be46f058714afab5d47b7bb700c1ff, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:11.204786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45bda8-7814-4bda-90c8-096e021aa92a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:11.285354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1009/2047] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:11 2022] PUT /identity/v3/projects/f03708a315eb41ee906b1e9d105e5f5a/users/c8be46f058714afab5d47b7bb700c1ff/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:11.290979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350b9d7-fa47-4029-b072-b84bb71286bf admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:11.302247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350b9d7-fa47-4029-b072-b84bb71286bf admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:11.372700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57921a-a33d-4464-a921-543e74427c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:11.373365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57921a-a33d-4464-a921-543e74427c72 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:11.373649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57921a-a33d-4464-a921-543e74427c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:11.373923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57921a-a33d-4464-a921-543e74427c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:11.374622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57921a-a33d-4464-a921-543e74427c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:11.376312 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57921a-a33d-4464-a921-543e74427c72 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:11.389832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1039/2048] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:11.396625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45bda8-7814-4bda-90c8-096e021aa92a admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:11.408355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45bda8-7814-4bda-90c8-096e021aa92a admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:11.414213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068203e-63dd-47f2-a70d-ecb1763fd1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:11.414964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068203e-63dd-47f2-a70d-ecb1763fd1cb admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:11.415243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068203e-63dd-47f2-a70d-ecb1763fd1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:11.415520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068203e-63dd-47f2-a70d-ecb1763fd1cb admin admin] PATH_INFO: `/v3/projects/f03708a315eb41ee906b1e9d105e5f5a/users/c8be46f058714afab5d47b7bb700c1ff/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:11.417685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068203e-63dd-47f2-a70d-ecb1763fd1cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03708a315eb41ee906b1e9d105e5f5a, user_id=c8be46f058714afab5d47b7bb700c1ff, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:11.421615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068203e-63dd-47f2-a70d-ecb1763fd1cb admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:11.440097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1010/2049] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:11 2022] PUT /identity/v3/projects/f03708a315eb41ee906b1e9d105e5f5a/users/c8be46f058714afab5d47b7bb700c1ff/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:11.453467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda084eb-c217-4f1a-868a-12e2252b9c56 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:11.453979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda084eb-c217-4f1a-868a-12e2252b9c56 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:11.454435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda084eb-c217-4f1a-868a-12e2252b9c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:11.493907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-bda084eb-c217-4f1a-868a-12e2252b9c56 None None] MFA Rules not processed for user `c8be46f058714afab5d47b7bb700c1ff`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:11.499159 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bda084eb-c217-4f1a-868a-12e2252b9c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:11.600669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1040/2050] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:11 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:11.615470 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068203e-63dd-47f2-a70d-ecb1763fd1cb admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:11.617736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2068203e-63dd-47f2-a70d-ecb1763fd1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:11.693133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:11.746108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068203e-63dd-47f2-a70d-ecb1763fd1cb admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:11.758525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddf391-1df2-475c-b9aa-44ad0d712a7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:11.759475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddf391-1df2-475c-b9aa-44ad0d712a7f service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:11.759915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddf391-1df2-475c-b9aa-44ad0d712a7f service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:11.760534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddf391-1df2-475c-b9aa-44ad0d712a7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:11.777355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddf391-1df2-475c-b9aa-44ad0d712a7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:11.779376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddf391-1df2-475c-b9aa-44ad0d712a7f service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:11.804208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1011/2051] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:11 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:28:11.968415 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda084eb-c217-4f1a-868a-12e2252b9c56 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:11.993190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:12.046229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda084eb-c217-4f1a-868a-12e2252b9c56 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:12.052181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:12.053086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:12.053430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:12.053765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] PATH_INFO: `/v3/users/4e18ecf2cbd8424d80c023bb2ed50a41` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:12.055364 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e18ecf2cbd8424d80c023bb2ed50a41)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:12.057954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:12.268921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e18ecf2cbd8424d80c023bb2ed50a41'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:12.269736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e18ecf2cbd8424d80c023bb2ed50a41'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:12.271186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e18ecf2cbd8424d80c023bb2ed50a41'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:12.384553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e18ecf2cbd8424d80c023bb2ed50a41'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:12.393620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e18ecf2cbd8424d80c023bb2ed50a41'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:12.417980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1041/2052] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:11 2022] DELETE /identity/v3/users/4e18ecf2cbd8424d80c023bb2ed50a41 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:12.423162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfe9b5-0c22-4e0c-89d7-5c78ebf02c4d None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:12.423600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfe9b5-0c22-4e0c-89d7-5c78ebf02c4d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:12.423967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfe9b5-0c22-4e0c-89d7-5c78ebf02c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:12.447372 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2dbfe9b5-0c22-4e0c-89d7-5c78ebf02c4d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:12.449013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2dbfe9b5-0c22-4e0c-89d7-5c78ebf02c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:12.525753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1012/2053] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:12.543026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:12.544602 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:12.637682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47b64f-f644-4f9f-8655-20c080be5f78 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:12.641983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3e7b6-dea3-4f69-a50e-4e9dfd7089c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:12.642735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3e7b6-dea3-4f69-a50e-4e9dfd7089c8 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:12.643011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3e7b6-dea3-4f69-a50e-4e9dfd7089c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:12.643282 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3e7b6-dea3-4f69-a50e-4e9dfd7089c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:12.654376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3e7b6-dea3-4f69-a50e-4e9dfd7089c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:12.656082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3e7b6-dea3-4f69-a50e-4e9dfd7089c8 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:12.668910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1042/2054] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:28:13.296581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfe9b5-0c22-4e0c-89d7-5c78ebf02c4d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:13.298739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2dbfe9b5-0c22-4e0c-89d7-5c78ebf02c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:13.323490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfe9b5-0c22-4e0c-89d7-5c78ebf02c4d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:13.328919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:13.329584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:13.329868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:13.330140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] PATH_INFO: `/v3/projects/c6837919a3f7439785e65ee360bbfb24` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:13.331583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6837919a3f7439785e65ee360bbfb24)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:13.334330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:13.784391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6837919a3f7439785e65ee360bbfb24', 'request_id': 'req-042567fb-d6f9-4d2e-b873-d224f6938313'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:13.792825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6837919a3f7439785e65ee360bbfb24', 'request_id': 'req-042567fb-d6f9-4d2e-b873-d224f6938313'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:13.961887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6837919a3f7439785e65ee360bbfb24', 'request_id': 'req-042567fb-d6f9-4d2e-b873-d224f6938313'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:13.983849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] The token cache is being invalidate because project c6837919a3f7439785e65ee360bbfb24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:13.984559 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:13.985729 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:13.988443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1013/2055] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:13 2022] DELETE /identity/v3/projects/c6837919a3f7439785e65ee360bbfb24 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:13.994115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3e7b6-dea3-4f69-a50e-4e9dfd7089c8 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:14.010772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3e7b6-dea3-4f69-a50e-4e9dfd7089c8 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:14.078950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:14.109790 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:14.110598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:14.110994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:14.111380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] PATH_INFO: `/v3/users/c8be46f058714afab5d47b7bb700c1ff` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:14.116116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8be46f058714afab5d47b7bb700c1ff)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:14.121673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:14.363875 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8be46f058714afab5d47b7bb700c1ff'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:14.364819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8be46f058714afab5d47b7bb700c1ff'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:14.365491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8be46f058714afab5d47b7bb700c1ff'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:14.447837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8be46f058714afab5d47b7bb700c1ff'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:14.456983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-aa26269d-47a7-4925-b478-8ee4333844ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8be46f058714afab5d47b7bb700c1ff'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:14.477499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1043/2056] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:13 2022] DELETE /identity/v3/users/c8be46f058714afab5d47b7bb700c1ff => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:15.137346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:15.139889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:15.161004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042567fb-d6f9-4d2e-b873-d224f6938313 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:15.232512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:15.244854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:15.248349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:15.248349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:15.248349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] PATH_INFO: `/v3/projects/f03708a315eb41ee906b1e9d105e5f5a` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:15.248349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f03708a315eb41ee906b1e9d105e5f5a)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:15.249378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:15.367354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f03708a315eb41ee906b1e9d105e5f5a', 'request_id': 'req-859e984a-a694-49a3-9eab-6df1eea11ad3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:15.375929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f03708a315eb41ee906b1e9d105e5f5a', 'request_id': 'req-859e984a-a694-49a3-9eab-6df1eea11ad3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:15.447425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f03708a315eb41ee906b1e9d105e5f5a', 'request_id': 'req-859e984a-a694-49a3-9eab-6df1eea11ad3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:15.460755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] The token cache is being invalidate because project f03708a315eb41ee906b1e9d105e5f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:15.461475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:15.462560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-859e984a-a694-49a3-9eab-6df1eea11ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:15.464569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1014/2057] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:15 2022] DELETE /identity/v3/projects/f03708a315eb41ee906b1e9d105e5f5a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:15.470904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf30da4-2ed7-41ae-896d-7df67d56a324 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:15.471543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf30da4-2ed7-41ae-896d-7df67d56a324 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:15.471901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf30da4-2ed7-41ae-896d-7df67d56a324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:15.517461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6cf30da4-2ed7-41ae-896d-7df67d56a324 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:15.520180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6cf30da4-2ed7-41ae-896d-7df67d56a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:15.627725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1044/2058] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:15.636083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341648f-89aa-42c8-abd8-35d1cb1e44d8 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:15.636379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341648f-89aa-42c8-abd8-35d1cb1e44d8 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:15.636636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341648f-89aa-42c8-abd8-35d1cb1e44d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:15.659516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d341648f-89aa-42c8-abd8-35d1cb1e44d8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:15.661592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d341648f-89aa-42c8-abd8-35d1cb1e44d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:15.678210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1015/2059] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:15.683634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf30da4-2ed7-41ae-896d-7df67d56a324 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:15.707283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf30da4-2ed7-41ae-896d-7df67d56a324 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:15.713012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76d432-8d5e-4500-95f3-5e5d28e0a8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:15.714442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d432-8d5e-4500-95f3-5e5d28e0a8ee admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:15.714897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d432-8d5e-4500-95f3-5e5d28e0a8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:15.715306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d432-8d5e-4500-95f3-5e5d28e0a8ee admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:15.716268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76d432-8d5e-4500-95f3-5e5d28e0a8ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:15.716674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76d432-8d5e-4500-95f3-5e5d28e0a8ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:15.719946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76d432-8d5e-4500-95f3-5e5d28e0a8ee admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:15.739458 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1045/2060] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:28:15.745281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341648f-89aa-42c8-abd8-35d1cb1e44d8 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:15.762864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341648f-89aa-42c8-abd8-35d1cb1e44d8 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:15.771739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f54efe-fcbd-4d15-b116-0f54131d28d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:15.772794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f54efe-fcbd-4d15-b116-0f54131d28d4 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:15.773309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f54efe-fcbd-4d15-b116-0f54131d28d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:15.773755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f54efe-fcbd-4d15-b116-0f54131d28d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:15.774974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f54efe-fcbd-4d15-b116-0f54131d28d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:15.778850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f54efe-fcbd-4d15-b116-0f54131d28d4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:15.878980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1016/2061] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:15 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:15.883664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76d432-8d5e-4500-95f3-5e5d28e0a8ee admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:15.895006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76d432-8d5e-4500-95f3-5e5d28e0a8ee admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:15.950518 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:15.967394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e670a-b47f-4019-a5bf-df69969b4125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:15.968025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e670a-b47f-4019-a5bf-df69969b4125 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:15.968304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e670a-b47f-4019-a5bf-df69969b4125 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:15.968573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e670a-b47f-4019-a5bf-df69969b4125 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:15.969220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e670a-b47f-4019-a5bf-df69969b4125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:15.971255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e670a-b47f-4019-a5bf-df69969b4125 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:16.099460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1046/2062] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:15 2022] POST /identity/v3/users => generated 288 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:16.104946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f54efe-fcbd-4d15-b116-0f54131d28d4 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:16.120539 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f54efe-fcbd-4d15-b116-0f54131d28d4 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:16.127234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c55c3-5dac-46f6-9a65-ccbcc46c6bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:16.128223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c55c3-5dac-46f6-9a65-ccbcc46c6bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:16.128722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c55c3-5dac-46f6-9a65-ccbcc46c6bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:16.129097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c55c3-5dac-46f6-9a65-ccbcc46c6bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:16.130178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c55c3-5dac-46f6-9a65-ccbcc46c6bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:16.132213 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c55c3-5dac-46f6-9a65-ccbcc46c6bd8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:16.147188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1017/2063] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:16.153712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e670a-b47f-4019-a5bf-df69969b4125 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:16.174039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e670a-b47f-4019-a5bf-df69969b4125 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:16.183659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7d49a-a304-4cf1-a9b1-ca8cb5ffaf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:16.184710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7d49a-a304-4cf1-a9b1-ca8cb5ffaf63 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:16.185201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7d49a-a304-4cf1-a9b1-ca8cb5ffaf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:16.185734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7d49a-a304-4cf1-a9b1-ca8cb5ffaf63 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:16.186949 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7d49a-a304-4cf1-a9b1-ca8cb5ffaf63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:16.190120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7d49a-a304-4cf1-a9b1-ca8cb5ffaf63 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:16.212122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1047/2064] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:16.217193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c55c3-5dac-46f6-9a65-ccbcc46c6bd8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:16.228112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c55c3-5dac-46f6-9a65-ccbcc46c6bd8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:16.233126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce477008-386c-471a-af76-5ce5879e4c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:16.233814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce477008-386c-471a-af76-5ce5879e4c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:16.234096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce477008-386c-471a-af76-5ce5879e4c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:16.234417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce477008-386c-471a-af76-5ce5879e4c99 admin admin] PATH_INFO: `/v3/projects/d23958845d514cb38c47acd052a41bf4/users/a29b657caec14252bc7e2637797f68f9/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:16.252557 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce477008-386c-471a-af76-5ce5879e4c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23958845d514cb38c47acd052a41bf4, user_id=a29b657caec14252bc7e2637797f68f9, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:16.256175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce477008-386c-471a-af76-5ce5879e4c99 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:16.345953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1018/2065] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:16 2022] PUT /identity/v3/projects/d23958845d514cb38c47acd052a41bf4/users/a29b657caec14252bc7e2637797f68f9/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:16.358981 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c01be4-d55f-4287-ac12-36aec4af1b3d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:16.359363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c01be4-d55f-4287-ac12-36aec4af1b3d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:16.359683 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c01be4-d55f-4287-ac12-36aec4af1b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:16.393349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-80c01be4-d55f-4287-ac12-36aec4af1b3d None None] MFA Rules not processed for user `a29b657caec14252bc7e2637797f68f9`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:16.401423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-80c01be4-d55f-4287-ac12-36aec4af1b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:16.474678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1048/2066] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:16.479105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce477008-386c-471a-af76-5ce5879e4c99 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:16.554185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce477008-386c-471a-af76-5ce5879e4c99 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:16.559806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dba5c6-7dc6-4e23-812d-af16146d2ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:16.560583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dba5c6-7dc6-4e23-812d-af16146d2ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:16.560933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dba5c6-7dc6-4e23-812d-af16146d2ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:16.561299 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dba5c6-7dc6-4e23-812d-af16146d2ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:16.562023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dba5c6-7dc6-4e23-812d-af16146d2ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:16.564175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dba5c6-7dc6-4e23-812d-af16146d2ab8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:16.690454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1019/2067] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:16 2022] POST /identity/v3/projects => generated 379 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:16.696111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c01be4-d55f-4287-ac12-36aec4af1b3d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:16.736232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c01be4-d55f-4287-ac12-36aec4af1b3d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:16.877352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b97cd-67ad-42b2-a4ce-04dd87c4da74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:16.878446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b97cd-67ad-42b2-a4ce-04dd87c4da74 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:16.878980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b97cd-67ad-42b2-a4ce-04dd87c4da74 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:16.879693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b97cd-67ad-42b2-a4ce-04dd87c4da74 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:16.880839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b97cd-67ad-42b2-a4ce-04dd87c4da74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:16.884553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b97cd-67ad-42b2-a4ce-04dd87c4da74 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:16.969304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1049/2068] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:16 2022] POST /identity/v3/users => generated 290 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:16.976498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dba5c6-7dc6-4e23-812d-af16146d2ab8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:16.984189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:16.990643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dba5c6-7dc6-4e23-812d-af16146d2ab8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:16.995808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3e158-eab4-4127-bb87-c1f679202620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:16.996441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3e158-eab4-4127-bb87-c1f679202620 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:16.996723 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3e158-eab4-4127-bb87-c1f679202620 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:16.996997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3e158-eab4-4127-bb87-c1f679202620 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:16.997653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3e158-eab4-4127-bb87-c1f679202620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:16.999354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3e158-eab4-4127-bb87-c1f679202620 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:17.012763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1020/2069] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:17.020823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b97cd-67ad-42b2-a4ce-04dd87c4da74 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:17.032904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b97cd-67ad-42b2-a4ce-04dd87c4da74 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:17.038583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9c5c7-0e6b-4927-a81c-22b34f1dfe8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:17.039208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9c5c7-0e6b-4927-a81c-22b34f1dfe8c admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:17.039486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9c5c7-0e6b-4927-a81c-22b34f1dfe8c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:17.039760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9c5c7-0e6b-4927-a81c-22b34f1dfe8c admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:17.040493 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9c5c7-0e6b-4927-a81c-22b34f1dfe8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:17.042135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9c5c7-0e6b-4927-a81c-22b34f1dfe8c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:17.054285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1050/2070] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:17.058507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3e158-eab4-4127-bb87-c1f679202620 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:17.073569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3e158-eab4-4127-bb87-c1f679202620 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:17.083183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fd409-39e3-49fd-b85d-c0c16b9cb02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:17.084311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fd409-39e3-49fd-b85d-c0c16b9cb02e admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:17.084773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fd409-39e3-49fd-b85d-c0c16b9cb02e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:17.085222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fd409-39e3-49fd-b85d-c0c16b9cb02e admin admin] PATH_INFO: `/v3/projects/55c11ea6b2cf4bfaa8e2b35f02c2ebc2/users/8b4cf3864bff4acebc372d4f51949487/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:17.112715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fd409-39e3-49fd-b85d-c0c16b9cb02e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c11ea6b2cf4bfaa8e2b35f02c2ebc2, user_id=8b4cf3864bff4acebc372d4f51949487, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:17.126921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fd409-39e3-49fd-b85d-c0c16b9cb02e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:17.209548 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1021/2071] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:17 2022] PUT /identity/v3/projects/55c11ea6b2cf4bfaa8e2b35f02c2ebc2/users/8b4cf3864bff4acebc372d4f51949487/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:17.227682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00128a8-5a5f-4f0d-91f6-f46e74f2e094 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:17.228028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00128a8-5a5f-4f0d-91f6-f46e74f2e094 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:17.228355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00128a8-5a5f-4f0d-91f6-f46e74f2e094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:17.265379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d00128a8-5a5f-4f0d-91f6-f46e74f2e094 None None] MFA Rules not processed for user `8b4cf3864bff4acebc372d4f51949487`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:17.267643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d00128a8-5a5f-4f0d-91f6-f46e74f2e094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:17.300438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:17.357858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1051/2072] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:17.362897 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fd409-39e3-49fd-b85d-c0c16b9cb02e admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:17.451108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fd409-39e3-49fd-b85d-c0c16b9cb02e admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:17.456334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670b52a-dc63-44d5-8f45-d6cec1f3f562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:17.456969 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670b52a-dc63-44d5-8f45-d6cec1f3f562 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:17.457769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670b52a-dc63-44d5-8f45-d6cec1f3f562 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:17.458041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670b52a-dc63-44d5-8f45-d6cec1f3f562 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:17.458739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670b52a-dc63-44d5-8f45-d6cec1f3f562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:17.460719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670b52a-dc63-44d5-8f45-d6cec1f3f562 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:17.564792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1022/2073] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:17 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:17.570060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00128a8-5a5f-4f0d-91f6-f46e74f2e094 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:17.582043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00128a8-5a5f-4f0d-91f6-f46e74f2e094 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:17.657575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d689d-990e-463c-9de9-a72e31b67a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:17.658451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d689d-990e-463c-9de9-a72e31b67a5d admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:17.658945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d689d-990e-463c-9de9-a72e31b67a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:17.659338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d689d-990e-463c-9de9-a72e31b67a5d admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:17.660190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d689d-990e-463c-9de9-a72e31b67a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:17.662980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d689d-990e-463c-9de9-a72e31b67a5d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:17.777295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1052/2074] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:17 2022] POST /identity/v3/users => generated 295 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:17.783126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670b52a-dc63-44d5-8f45-d6cec1f3f562 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:17.809988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670b52a-dc63-44d5-8f45-d6cec1f3f562 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:17.820184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8996c-ec0b-4c55-9375-470e32a43c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:17.820184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8996c-ec0b-4c55-9375-470e32a43c13 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:17.820184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8996c-ec0b-4c55-9375-470e32a43c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:17.820184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8996c-ec0b-4c55-9375-470e32a43c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:17.820927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8996c-ec0b-4c55-9375-470e32a43c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:17.820927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8996c-ec0b-4c55-9375-470e32a43c13 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:17.835984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1023/2075] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:17.842445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d689d-990e-463c-9de9-a72e31b67a5d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:17.854626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d689d-990e-463c-9de9-a72e31b67a5d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:17.861262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31118038-facb-4116-8b5c-96fe7e7ebfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:17.862336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31118038-facb-4116-8b5c-96fe7e7ebfb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:17.863106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31118038-facb-4116-8b5c-96fe7e7ebfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:17.864480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31118038-facb-4116-8b5c-96fe7e7ebfb7 admin admin] PATH_INFO: `/v3/projects/a6ed7ccbaf04455ca32189dd423fbd4c/users/e48318df7fb2459c817b89a7805b8e81/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:17.886247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31118038-facb-4116-8b5c-96fe7e7ebfb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ed7ccbaf04455ca32189dd423fbd4c, user_id=e48318df7fb2459c817b89a7805b8e81, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:17.889465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31118038-facb-4116-8b5c-96fe7e7ebfb7 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:17.981991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1053/2076] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:17 2022] PUT /identity/v3/projects/a6ed7ccbaf04455ca32189dd423fbd4c/users/e48318df7fb2459c817b89a7805b8e81/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:17.989989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8996c-ec0b-4c55-9375-470e32a43c13 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:18.010814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8996c-ec0b-4c55-9375-470e32a43c13 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:18.143112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c354a63-4d5c-4ba9-b85c-cc64e7227613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:18.152590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354a63-4d5c-4ba9-b85c-cc64e7227613 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:18.153474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354a63-4d5c-4ba9-b85c-cc64e7227613 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:18.154668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354a63-4d5c-4ba9-b85c-cc64e7227613 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:18.157529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c354a63-4d5c-4ba9-b85c-cc64e7227613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:18.160652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c354a63-4d5c-4ba9-b85c-cc64e7227613 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:18.194038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31118038-facb-4116-8b5c-96fe7e7ebfb7 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:18.201466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1024/2077] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:18.217641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31118038-facb-4116-8b5c-96fe7e7ebfb7 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:18.231897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d1007-2c51-4d76-b8de-acb82e31b5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:18.233794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d1007-2c51-4d76-b8de-acb82e31b5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:18.234649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d1007-2c51-4d76-b8de-acb82e31b5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:18.235420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d1007-2c51-4d76-b8de-acb82e31b5fb admin admin] PATH_INFO: `/v3/projects/a6ed7ccbaf04455ca32189dd423fbd4c/users/e48318df7fb2459c817b89a7805b8e81/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:18.239894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d1007-2c51-4d76-b8de-acb82e31b5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ed7ccbaf04455ca32189dd423fbd4c, user_id=e48318df7fb2459c817b89a7805b8e81, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:18.244211 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d1007-2c51-4d76-b8de-acb82e31b5fb admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:18.264701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1054/2078] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:18 2022] PUT /identity/v3/projects/a6ed7ccbaf04455ca32189dd423fbd4c/users/e48318df7fb2459c817b89a7805b8e81/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:18.284910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a686b8b-d2af-4d03-be5c-01752bb5d2db None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:18.287234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a686b8b-d2af-4d03-be5c-01752bb5d2db None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:18.299644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a686b8b-d2af-4d03-be5c-01752bb5d2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:18.338595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-9a686b8b-d2af-4d03-be5c-01752bb5d2db None None] MFA Rules not processed for user `e48318df7fb2459c817b89a7805b8e81`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:18.339866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9a686b8b-d2af-4d03-be5c-01752bb5d2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:18.373317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:18.437819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1025/2079] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:18.455703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d1007-2c51-4d76-b8de-acb82e31b5fb admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:18.457728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6e4d1007-2c51-4d76-b8de-acb82e31b5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:18.531723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d1007-2c51-4d76-b8de-acb82e31b5fb admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:18.537888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598ae1a-7ef1-45e0-bc90-428b0c3ce1ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:18.538576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598ae1a-7ef1-45e0-bc90-428b0c3ce1ca service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:18.538864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598ae1a-7ef1-45e0-bc90-428b0c3ce1ca service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:18.539130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598ae1a-7ef1-45e0-bc90-428b0c3ce1ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:18.549552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598ae1a-7ef1-45e0-bc90-428b0c3ce1ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:18.551736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598ae1a-7ef1-45e0-bc90-428b0c3ce1ca service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:18.562640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1055/2080] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:28:18.683595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a686b8b-d2af-4d03-be5c-01752bb5d2db None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:18.685143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9a686b8b-d2af-4d03-be5c-01752bb5d2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:18.699359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a686b8b-d2af-4d03-be5c-01752bb5d2db None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:18.703026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf63e6d-ba76-4234-9470-fc1fbfc1d8bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:18.703649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf63e6d-ba76-4234-9470-fc1fbfc1d8bf service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:18.703926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf63e6d-ba76-4234-9470-fc1fbfc1d8bf service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:18.704210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf63e6d-ba76-4234-9470-fc1fbfc1d8bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:18.789852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf63e6d-ba76-4234-9470-fc1fbfc1d8bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:18.793065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf63e6d-ba76-4234-9470-fc1fbfc1d8bf service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:18.811711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1026/2081] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:28:28.557732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:28.803494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=45 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:34.004749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598ae1a-7ef1-45e0-bc90-428b0c3ce1ca service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:34.117419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598ae1a-7ef1-45e0-bc90-428b0c3ce1ca service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:34.122945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:34.123687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:34.124063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:34.124447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] PATH_INFO: `/v3/users/a29b657caec14252bc7e2637797f68f9` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:34.125794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29b657caec14252bc7e2637797f68f9)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:34.127816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:34.415833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29b657caec14252bc7e2637797f68f9'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:34.416963 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29b657caec14252bc7e2637797f68f9'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:34.418375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29b657caec14252bc7e2637797f68f9'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:34.516530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29b657caec14252bc7e2637797f68f9'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:34.534871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29b657caec14252bc7e2637797f68f9'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:34.561028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1056/2082] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:34 2022] DELETE /identity/v3/users/a29b657caec14252bc7e2637797f68f9 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:34.566861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a11701-7f90-4929-9b1a-6ab435e124dd None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:34.567746 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a11701-7f90-4929-9b1a-6ab435e124dd None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:34.567746 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a11701-7f90-4929-9b1a-6ab435e124dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:34.589462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c7a11701-7f90-4929-9b1a-6ab435e124dd None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:34.591093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c7a11701-7f90-4929-9b1a-6ab435e124dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:34.670449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1027/2083] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:34.682898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:34.684625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:34.765780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ac9d2-aef1-4b68-9993-5150b866e173 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:34.770512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27d94a-6013-46fb-af94-1d30037f42f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:34.771079 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27d94a-6013-46fb-af94-1d30037f42f1 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:34.771356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27d94a-6013-46fb-af94-1d30037f42f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:34.771742 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27d94a-6013-46fb-af94-1d30037f42f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:34.782440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27d94a-6013-46fb-af94-1d30037f42f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:34.783970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27d94a-6013-46fb-af94-1d30037f42f1 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:34.795700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1057/2084] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:28:35.352826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a11701-7f90-4929-9b1a-6ab435e124dd None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:35.355675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c7a11701-7f90-4929-9b1a-6ab435e124dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:35.377670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a11701-7f90-4929-9b1a-6ab435e124dd None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:35.385350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:35.386304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:35.387029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:35.387494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] PATH_INFO: `/v3/projects/d23958845d514cb38c47acd052a41bf4` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:35.391439 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23958845d514cb38c47acd052a41bf4)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:35.395541 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:35.570599 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23958845d514cb38c47acd052a41bf4', 'request_id': 'req-f07cd829-e09b-46e9-9da3-047a680c0fb0'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:35.587946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23958845d514cb38c47acd052a41bf4', 'request_id': 'req-f07cd829-e09b-46e9-9da3-047a680c0fb0'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:35.716370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23958845d514cb38c47acd052a41bf4', 'request_id': 'req-f07cd829-e09b-46e9-9da3-047a680c0fb0'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:35.744783 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] The token cache is being invalidate because project d23958845d514cb38c47acd052a41bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:35.745330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:35.746155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:35.748062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1028/2085] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:35 2022] DELETE /identity/v3/projects/d23958845d514cb38c47acd052a41bf4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:35.757773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27d94a-6013-46fb-af94-1d30037f42f1 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:35.770966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27d94a-6013-46fb-af94-1d30037f42f1 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:35.794702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:35.852993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:35.854013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:35.854198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:35.854532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] PATH_INFO: `/v3/users/8b4cf3864bff4acebc372d4f51949487` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:35.855976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4cf3864bff4acebc372d4f51949487)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:35.858732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:36.099623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4cf3864bff4acebc372d4f51949487'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:36.100378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4cf3864bff4acebc372d4f51949487'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:36.101100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4cf3864bff4acebc372d4f51949487'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:36.156099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4cf3864bff4acebc372d4f51949487'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:36.166392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4cf3864bff4acebc372d4f51949487'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:36.186664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1058/2086] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:35 2022] DELETE /identity/v3/users/8b4cf3864bff4acebc372d4f51949487 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:36.705444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:36.708825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:36.733401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cd829-e09b-46e9-9da3-047a680c0fb0 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:36.826174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:36.827211 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:36.827580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:36.827939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] PATH_INFO: `/v3/projects/55c11ea6b2cf4bfaa8e2b35f02c2ebc2` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:36.829730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c11ea6b2cf4bfaa8e2b35f02c2ebc2)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:36.833587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:36.985000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:37.029993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c11ea6b2cf4bfaa8e2b35f02c2ebc2', 'request_id': 'req-a542c67b-11ad-4798-9e0b-06b11c3a53c1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:37.039831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c11ea6b2cf4bfaa8e2b35f02c2ebc2', 'request_id': 'req-a542c67b-11ad-4798-9e0b-06b11c3a53c1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:37.116282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c11ea6b2cf4bfaa8e2b35f02c2ebc2', 'request_id': 'req-a542c67b-11ad-4798-9e0b-06b11c3a53c1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:37.135500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] The token cache is being invalidate because project 55c11ea6b2cf4bfaa8e2b35f02c2ebc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:37.137063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:37.138084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:37.143586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1029/2087] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:36 2022] DELETE /identity/v3/projects/55c11ea6b2cf4bfaa8e2b35f02c2ebc2 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:37.146122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:37.147938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:37.173628 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d5c44-0540-4084-8bb9-041721365ac2 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:37.257507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:37.258674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:37.259181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:37.259657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] PATH_INFO: `/v3/users/e48318df7fb2459c817b89a7805b8e81` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:37.261769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e48318df7fb2459c817b89a7805b8e81)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:37.266065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:37.510936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:37.513812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e48318df7fb2459c817b89a7805b8e81'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:37.514935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e48318df7fb2459c817b89a7805b8e81'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:37.515580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e48318df7fb2459c817b89a7805b8e81'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:37.597216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e48318df7fb2459c817b89a7805b8e81'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:37.607598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e48318df7fb2459c817b89a7805b8e81'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:37.647655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1059/2088] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:37 2022] DELETE /identity/v3/users/e48318df7fb2459c817b89a7805b8e81 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:38.255170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:38.257356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:38.283093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542c67b-11ad-4798-9e0b-06b11c3a53c1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:38.384759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:38.386890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:38.403023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:38.404416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:38.404888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:38.405652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] PATH_INFO: `/v3/users/601be948ace540caac3e54af70092b3e` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:38.408483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=601be948ace540caac3e54af70092b3e)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:38.413594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:38.514859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6fc6c-81e7-4f30-9bee-91e22060c56e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:38.520876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:38.521544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:38.521822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:38.522094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] PATH_INFO: `/v3/projects/a6ed7ccbaf04455ca32189dd423fbd4c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:38.523440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ed7ccbaf04455ca32189dd423fbd4c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:38.525556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:38.700563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '601be948ace540caac3e54af70092b3e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:38.712137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '601be948ace540caac3e54af70092b3e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:38.713230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '601be948ace540caac3e54af70092b3e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:38.714827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ed7ccbaf04455ca32189dd423fbd4c', 'request_id': 'req-cde9c2b3-258b-460e-bcdc-79f603c892e1'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:38.728249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ed7ccbaf04455ca32189dd423fbd4c', 'request_id': 'req-cde9c2b3-258b-460e-bcdc-79f603c892e1'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:38.728753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '601be948ace540caac3e54af70092b3e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:38.745772 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-90065bdd-9d28-4d2c-b50d-6756b55ae990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '601be948ace540caac3e54af70092b3e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:38.748894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ed7ccbaf04455ca32189dd423fbd4c', 'request_id': 'req-cde9c2b3-258b-460e-bcdc-79f603c892e1'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:38.825979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] The token cache is being invalidate because project a6ed7ccbaf04455ca32189dd423fbd4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:38.828577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:38.828577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-cde9c2b3-258b-460e-bcdc-79f603c892e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:38.829653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1060/2089] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:38 2022] DELETE /identity/v3/projects/a6ed7ccbaf04455ca32189dd423fbd4c => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:38.836711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dab10-3663-4d4a-925c-ccf6c83f5d43 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:38.837358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dab10-3663-4d4a-925c-ccf6c83f5d43 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:38.837636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dab10-3663-4d4a-925c-ccf6c83f5d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:38.859870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-678dab10-3663-4d4a-925c-ccf6c83f5d43 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:38.861140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-678dab10-3663-4d4a-925c-ccf6c83f5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:38.921456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:38.922750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1030/2090] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:38 2022] DELETE /identity/v3/users/601be948ace540caac3e54af70092b3e => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:38.925336 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62c76f-6056-4ac3-a1a3-8806b8b8a1a3 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:38.927524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62c76f-6056-4ac3-a1a3-8806b8b8a1a3 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:38.928052 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62c76f-6056-4ac3-a1a3-8806b8b8a1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:38.941004 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1061/2091] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:38.953880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d730d-a56d-4a19-81d9-0171af084ed6 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:38.954376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d730d-a56d-4a19-81d9-0171af084ed6 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:38.954858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d730d-a56d-4a19-81d9-0171af084ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:38.961336 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-4f62c76f-6056-4ac3-a1a3-8806b8b8a1a3 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:38.963464 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4f62c76f-6056-4ac3-a1a3-8806b8b8a1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:38.987777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c65d730d-a56d-4a19-81d9-0171af084ed6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:38.990238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c65d730d-a56d-4a19-81d9-0171af084ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:39.040237 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:39.099152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1062/2092] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:39.100436 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1031/2093] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:39.104853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d730d-a56d-4a19-81d9-0171af084ed6 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:39.108973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62c76f-6056-4ac3-a1a3-8806b8b8a1a3 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:39.111451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4f62c76f-6056-4ac3-a1a3-8806b8b8a1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:39.115596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d730d-a56d-4a19-81d9-0171af084ed6 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:39.122372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067f8b7-e176-4062-90fc-9e7ca04de757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:39.123170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067f8b7-e176-4062-90fc-9e7ca04de757 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:39.123607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067f8b7-e176-4062-90fc-9e7ca04de757 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:39.123935 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067f8b7-e176-4062-90fc-9e7ca04de757 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:39.124727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067f8b7-e176-4062-90fc-9e7ca04de757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:39.125144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067f8b7-e176-4062-90fc-9e7ca04de757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:39.128039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067f8b7-e176-4062-90fc-9e7ca04de757 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:39.151273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1063/2094] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:28:39.156358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067f8b7-e176-4062-90fc-9e7ca04de757 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:39.174432 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067f8b7-e176-4062-90fc-9e7ca04de757 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:39.185814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f008ad6-b621-4713-b5a7-bb40ea197187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:39.187456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f008ad6-b621-4713-b5a7-bb40ea197187 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:39.187928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f008ad6-b621-4713-b5a7-bb40ea197187 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:39.188416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f008ad6-b621-4713-b5a7-bb40ea197187 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:39.189770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f008ad6-b621-4713-b5a7-bb40ea197187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:39.193778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f008ad6-b621-4713-b5a7-bb40ea197187 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:39.202270 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62c76f-6056-4ac3-a1a3-8806b8b8a1a3 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:39.208947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519c044-f373-4daf-8201-e0ca81095ad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:39.210094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519c044-f373-4daf-8201-e0ca81095ad9 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:39.211650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519c044-f373-4daf-8201-e0ca81095ad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:39.211650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519c044-f373-4daf-8201-e0ca81095ad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:39.224248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519c044-f373-4daf-8201-e0ca81095ad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:39.226361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519c044-f373-4daf-8201-e0ca81095ad9 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:39.237659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1032/2095] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:28:39.377833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1064/2096] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:39 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:39.383396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519c044-f373-4daf-8201-e0ca81095ad9 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:39.419990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519c044-f373-4daf-8201-e0ca81095ad9 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:39.580308 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3aed9-840c-44f8-bb0c-e919449ad6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:39.581774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3aed9-840c-44f8-bb0c-e919449ad6fa admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:39.582392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3aed9-840c-44f8-bb0c-e919449ad6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:39.583041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3aed9-840c-44f8-bb0c-e919449ad6fa admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:39.584346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3aed9-840c-44f8-bb0c-e919449ad6fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:39.589044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3aed9-840c-44f8-bb0c-e919449ad6fa admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:39.693681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1033/2097] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:39 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:39.699519 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f008ad6-b621-4713-b5a7-bb40ea197187 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:39.713897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f008ad6-b621-4713-b5a7-bb40ea197187 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:39.724992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fb5b0-ae40-4ae9-a97b-827f25153098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:39.726433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fb5b0-ae40-4ae9-a97b-827f25153098 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:39.727484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fb5b0-ae40-4ae9-a97b-827f25153098 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:39.728219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fb5b0-ae40-4ae9-a97b-827f25153098 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:39.730553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fb5b0-ae40-4ae9-a97b-827f25153098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:39.733533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fb5b0-ae40-4ae9-a97b-827f25153098 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:39.750697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1065/2098] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:39.756736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3aed9-840c-44f8-bb0c-e919449ad6fa admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:39.781535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3aed9-840c-44f8-bb0c-e919449ad6fa admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:39.791254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a27bd-7d4f-4a35-8de3-8df1c27e7128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:39.792178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a27bd-7d4f-4a35-8de3-8df1c27e7128 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:39.794257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a27bd-7d4f-4a35-8de3-8df1c27e7128 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:39.794257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a27bd-7d4f-4a35-8de3-8df1c27e7128 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:39.794257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a27bd-7d4f-4a35-8de3-8df1c27e7128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:39.801126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a27bd-7d4f-4a35-8de3-8df1c27e7128 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:39.818543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1034/2099] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:39.820472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fb5b0-ae40-4ae9-a97b-827f25153098 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:39.840882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fb5b0-ae40-4ae9-a97b-827f25153098 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:39.849348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855353c4-68cf-4a85-bd5d-570765fb09ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:39.850517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855353c4-68cf-4a85-bd5d-570765fb09ce admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:39.851007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855353c4-68cf-4a85-bd5d-570765fb09ce admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:39.851474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855353c4-68cf-4a85-bd5d-570765fb09ce admin admin] PATH_INFO: `/v3/projects/b231da192b1b4d88b156e5b23f773370/users/1d60629f5bfa45deb6cc1d121bc08998/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:39.873378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a27bd-7d4f-4a35-8de3-8df1c27e7128 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:39.875441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8f4a27bd-7d4f-4a35-8de3-8df1c27e7128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:39.900954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855353c4-68cf-4a85-bd5d-570765fb09ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b231da192b1b4d88b156e5b23f773370, user_id=1d60629f5bfa45deb6cc1d121bc08998, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:39.901274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a27bd-7d4f-4a35-8de3-8df1c27e7128 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:39.907469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855353c4-68cf-4a85-bd5d-570765fb09ce admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:39.913541 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:39.914698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:39.914997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:39.915272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] PATH_INFO: `/v3/projects/c6751708050444a4a04f19a2072a4c64` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:39.941133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6751708050444a4a04f19a2072a4c64)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:39.941133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:40.022344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1066/2100] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:39 2022] PUT /identity/v3/projects/b231da192b1b4d88b156e5b23f773370/users/1d60629f5bfa45deb6cc1d121bc08998/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:40.037181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce828ed-3a9e-4ffb-ba8c-89ad752948d1 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:40.037452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce828ed-3a9e-4ffb-ba8c-89ad752948d1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:40.037779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce828ed-3a9e-4ffb-ba8c-89ad752948d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:40.087752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6751708050444a4a04f19a2072a4c64', 'request_id': 'req-c3040fa8-ba44-49c3-85fa-72c1261ea836'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:40.089174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-cce828ed-3a9e-4ffb-ba8c-89ad752948d1 None None] MFA Rules not processed for user `1d60629f5bfa45deb6cc1d121bc08998`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:40.090236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:40.091322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-cce828ed-3a9e-4ffb-ba8c-89ad752948d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:40.104190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6751708050444a4a04f19a2072a4c64', 'request_id': 'req-c3040fa8-ba44-49c3-85fa-72c1261ea836'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:40.173389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1067/2101] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:40.178173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce828ed-3a9e-4ffb-ba8c-89ad752948d1 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:40.197450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:40.214866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6751708050444a4a04f19a2072a4c64', 'request_id': 'req-c3040fa8-ba44-49c3-85fa-72c1261ea836'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:40.252041 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] The token cache is being invalidate because project c6751708050444a4a04f19a2072a4c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:40.252752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:40.254945 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:40.258659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1035/2102] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:39 2022] DELETE /identity/v3/projects/c6751708050444a4a04f19a2072a4c64 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:40.324334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce828ed-3a9e-4ffb-ba8c-89ad752948d1 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:40.337736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ab08f-c323-4bfe-b097-1117ccea2236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:40.339475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ab08f-c323-4bfe-b097-1117ccea2236 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:40.340330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ab08f-c323-4bfe-b097-1117ccea2236 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:40.340816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ab08f-c323-4bfe-b097-1117ccea2236 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:40.342107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ab08f-c323-4bfe-b097-1117ccea2236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:40.346764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ab08f-c323-4bfe-b097-1117ccea2236 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:40.492441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1068/2103] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:40 2022] POST /identity/v3/projects => generated 383 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:40.497155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:40.501081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:40.528553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3040fa8-ba44-49c3-85fa-72c1261ea836 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:40.656184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899db03-69cc-4192-a758-b16e8d8c0a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:40.656865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899db03-69cc-4192-a758-b16e8d8c0a26 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:40.657129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899db03-69cc-4192-a758-b16e8d8c0a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:40.657406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899db03-69cc-4192-a758-b16e8d8c0a26 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:40.658075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899db03-69cc-4192-a758-b16e8d8c0a26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:40.660682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899db03-69cc-4192-a758-b16e8d8c0a26 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:40.781565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1036/2104] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:40 2022] POST /identity/v3/users => generated 298 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:40.786328 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ab08f-c323-4bfe-b097-1117ccea2236 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:40.798564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ab08f-c323-4bfe-b097-1117ccea2236 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:40.805368 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169556fd-d375-4d9c-b899-1a860b7f0c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:40.806329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169556fd-d375-4d9c-b899-1a860b7f0c3b admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:40.806811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169556fd-d375-4d9c-b899-1a860b7f0c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:40.807269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169556fd-d375-4d9c-b899-1a860b7f0c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:40.811711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169556fd-d375-4d9c-b899-1a860b7f0c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:40.811711 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169556fd-d375-4d9c-b899-1a860b7f0c3b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:40.826145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1069/2105] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:40.830911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899db03-69cc-4192-a758-b16e8d8c0a26 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:40.846534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899db03-69cc-4192-a758-b16e8d8c0a26 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:40.854028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c557c9-ae1d-48a2-b76f-276d79187e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:40.855331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c557c9-ae1d-48a2-b76f-276d79187e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:40.855521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c557c9-ae1d-48a2-b76f-276d79187e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:40.855950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c557c9-ae1d-48a2-b76f-276d79187e45 admin admin] PATH_INFO: `/v3/projects/465a192f9f2b4c95a4a6a3e9d85759c6/users/d41d74708c6544d3a89ea6f08d0fc700/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:40.883269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c557c9-ae1d-48a2-b76f-276d79187e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=465a192f9f2b4c95a4a6a3e9d85759c6, user_id=d41d74708c6544d3a89ea6f08d0fc700, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:40.886117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c557c9-ae1d-48a2-b76f-276d79187e45 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:41.040811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1037/2106] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:40 2022] PUT /identity/v3/projects/465a192f9f2b4c95a4a6a3e9d85759c6/users/d41d74708c6544d3a89ea6f08d0fc700/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:41.045652 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169556fd-d375-4d9c-b899-1a860b7f0c3b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:41.060249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169556fd-d375-4d9c-b899-1a860b7f0c3b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:41.182333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1e08-9858-4f16-846b-02504c982679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:41.183993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1e08-9858-4f16-846b-02504c982679 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:41.184394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1e08-9858-4f16-846b-02504c982679 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:41.184880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1e08-9858-4f16-846b-02504c982679 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:41.186103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de1e08-9858-4f16-846b-02504c982679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:41.189479 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de1e08-9858-4f16-846b-02504c982679 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:41.209360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1070/2107] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:41.216134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c557c9-ae1d-48a2-b76f-276d79187e45 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:41.236258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c557c9-ae1d-48a2-b76f-276d79187e45 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:41.245372 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64090df-3727-48c5-996f-80f4eb0874b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:41.248150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64090df-3727-48c5-996f-80f4eb0874b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:41.248699 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64090df-3727-48c5-996f-80f4eb0874b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:41.249198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64090df-3727-48c5-996f-80f4eb0874b7 admin admin] PATH_INFO: `/v3/projects/465a192f9f2b4c95a4a6a3e9d85759c6/users/d41d74708c6544d3a89ea6f08d0fc700/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:41.253434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64090df-3727-48c5-996f-80f4eb0874b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=465a192f9f2b4c95a4a6a3e9d85759c6, user_id=d41d74708c6544d3a89ea6f08d0fc700, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:41.259891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64090df-3727-48c5-996f-80f4eb0874b7 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:41.278981 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1038/2108] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:41 2022] PUT /identity/v3/projects/465a192f9f2b4c95a4a6a3e9d85759c6/users/d41d74708c6544d3a89ea6f08d0fc700/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:41.299623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8211554-e06d-423f-ba4b-de84b933ac90 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:41.300018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8211554-e06d-423f-ba4b-de84b933ac90 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:41.300342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8211554-e06d-423f-ba4b-de84b933ac90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:41.359742 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c8211554-e06d-423f-ba4b-de84b933ac90 None None] MFA Rules not processed for user `d41d74708c6544d3a89ea6f08d0fc700`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:41.363952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c8211554-e06d-423f-ba4b-de84b933ac90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:41.516407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1071/2109] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:41.533080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64090df-3727-48c5-996f-80f4eb0874b7 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:41.538351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a64090df-3727-48c5-996f-80f4eb0874b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:41.710196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64090df-3727-48c5-996f-80f4eb0874b7 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:41.717785 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e7c731-45aa-40d9-b1b2-61f48a3a266b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:41.719017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7c731-45aa-40d9-b1b2-61f48a3a266b service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:41.719879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7c731-45aa-40d9-b1b2-61f48a3a266b service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:41.720376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7c731-45aa-40d9-b1b2-61f48a3a266b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:41.745005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e7c731-45aa-40d9-b1b2-61f48a3a266b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:41.748369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e7c731-45aa-40d9-b1b2-61f48a3a266b service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:41.775420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1039/2110] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:28:43.612473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8211554-e06d-423f-ba4b-de84b933ac90 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:43.638687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:43.757137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8211554-e06d-423f-ba4b-de84b933ac90 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:43.766338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:43.767658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:43.768130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:43.768598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] PATH_INFO: `/v3/users/1d60629f5bfa45deb6cc1d121bc08998` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:43.771766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d60629f5bfa45deb6cc1d121bc08998)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:43.777214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:44.127017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d60629f5bfa45deb6cc1d121bc08998'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:44.129331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d60629f5bfa45deb6cc1d121bc08998'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:44.131791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d60629f5bfa45deb6cc1d121bc08998'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:44.241859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d60629f5bfa45deb6cc1d121bc08998'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:44.253268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d60629f5bfa45deb6cc1d121bc08998'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:44.287702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1072/2111] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:43 2022] DELETE /identity/v3/users/1d60629f5bfa45deb6cc1d121bc08998 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:44.293021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cffac0-1d91-47b3-bd14-94043940bacc None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:44.293838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cffac0-1d91-47b3-bd14-94043940bacc None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:44.294073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cffac0-1d91-47b3-bd14-94043940bacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:44.322990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-44cffac0-1d91-47b3-bd14-94043940bacc None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:44.325819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-44cffac0-1d91-47b3-bd14-94043940bacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:44.446720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:44.474767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1040/2112] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:44.492143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:44.494387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:44.667474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb00c-42f1-4238-afbd-dee2d80842c0 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:44.667474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cecb5-2eab-40c7-8c61-efe1b4a94d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:44.667474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cecb5-2eab-40c7-8c61-efe1b4a94d7b service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:44.667474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cecb5-2eab-40c7-8c61-efe1b4a94d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:44.670901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cecb5-2eab-40c7-8c61-efe1b4a94d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:44.670901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4cecb5-2eab-40c7-8c61-efe1b4a94d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:44.670901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4cecb5-2eab-40c7-8c61-efe1b4a94d7b service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:44.692727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1073/2113] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:28:44.939536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cffac0-1d91-47b3-bd14-94043940bacc None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:44.941074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-44cffac0-1d91-47b3-bd14-94043940bacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:44.963877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cffac0-1d91-47b3-bd14-94043940bacc None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:44.972886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:44.979796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:44.979796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:44.979796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] PATH_INFO: `/v3/users/9eedc633493b478a9009f7bce07696a8` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:44.979796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eedc633493b478a9009f7bce07696a8)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:44.979796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:45.232352 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eedc633493b478a9009f7bce07696a8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:45.233394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eedc633493b478a9009f7bce07696a8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:45.234178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eedc633493b478a9009f7bce07696a8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:45.248494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eedc633493b478a9009f7bce07696a8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:45.272170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eedc633493b478a9009f7bce07696a8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:45.311662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cecb5-2eab-40c7-8c61-efe1b4a94d7b service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:45.314109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ec4cecb5-2eab-40c7-8c61-efe1b4a94d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:45.338263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cecb5-2eab-40c7-8c61-efe1b4a94d7b service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:45.347707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:45.347920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:45.348428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:45.348734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] PATH_INFO: `/v3/projects/b231da192b1b4d88b156e5b23f773370` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:45.350138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b231da192b1b4d88b156e5b23f773370)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:45.352290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:45.387510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:45.413155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1041/2114] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:44 2022] DELETE /identity/v3/users/9eedc633493b478a9009f7bce07696a8 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:45.579897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b231da192b1b4d88b156e5b23f773370', 'request_id': 'req-ec7c4ca4-98da-495d-9e42-f0b13672abb4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:45.590998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b231da192b1b4d88b156e5b23f773370', 'request_id': 'req-ec7c4ca4-98da-495d-9e42-f0b13672abb4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:45.600782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b231da192b1b4d88b156e5b23f773370', 'request_id': 'req-ec7c4ca4-98da-495d-9e42-f0b13672abb4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:45.716768 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] The token cache is being invalidate because project b231da192b1b4d88b156e5b23f773370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:45.717541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:45.717853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:45.719499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1074/2115] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:45 2022] DELETE /identity/v3/projects/b231da192b1b4d88b156e5b23f773370 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:45.725072 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:45.726773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:45.761249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:45.764119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:45.813244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8c6b2-922f-4a64-80da-15629f44108e admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:45.886502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:45.890721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c4ca4-98da-495d-9e42-f0b13672abb4 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:45.896606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:45.903529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:45.903529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:45.903529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] PATH_INFO: `/v3/projects/778299a985204da18b35d86ed8c4c77d` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:45.903529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=778299a985204da18b35d86ed8c4c77d)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:45.903529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:45.903992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:45.904473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:45.904976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:45.905462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] PATH_INFO: `/v3/users/d41d74708c6544d3a89ea6f08d0fc700` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:45.907648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41d74708c6544d3a89ea6f08d0fc700)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:45.912353 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:46.114796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778299a985204da18b35d86ed8c4c77d', 'request_id': 'req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:46.115503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41d74708c6544d3a89ea6f08d0fc700'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:46.116765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41d74708c6544d3a89ea6f08d0fc700'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:46.117667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41d74708c6544d3a89ea6f08d0fc700'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:46.126162 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778299a985204da18b35d86ed8c4c77d', 'request_id': 'req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:46.130429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41d74708c6544d3a89ea6f08d0fc700'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:46.134769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778299a985204da18b35d86ed8c4c77d', 'request_id': 'req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:46.147643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6388cef3-ec10-4c65-9773-448626392c0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41d74708c6544d3a89ea6f08d0fc700'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:46.237059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] The token cache is being invalidate because project 778299a985204da18b35d86ed8c4c77d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:46.237512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:46.238902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b95f6b71-9af4-47a5-be71-66ad6b2a0a93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:46.240628 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1075/2116] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:45 2022] DELETE /identity/v3/projects/778299a985204da18b35d86ed8c4c77d => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:46.245895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ff057-01a6-421f-b5a3-73c03deaa36c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:46.246150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ff057-01a6-421f-b5a3-73c03deaa36c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:46.246403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ff057-01a6-421f-b5a3-73c03deaa36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:46.269241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-843ff057-01a6-421f-b5a3-73c03deaa36c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:46.270526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-843ff057-01a6-421f-b5a3-73c03deaa36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:46.321318 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:46.340227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1042/2117] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:45 2022] DELETE /identity/v3/users/d41d74708c6544d3a89ea6f08d0fc700 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:46.368640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1076/2118] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:46.373710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d2f6-0acf-4d29-a9ce-444a00eb34a7 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:46.374210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d2f6-0acf-4d29-a9ce-444a00eb34a7 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:46.375210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d2f6-0acf-4d29-a9ce-444a00eb34a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:46.411664 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1fd3d2f6-0acf-4d29-a9ce-444a00eb34a7 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:46.413765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1fd3d2f6-0acf-4d29-a9ce-444a00eb34a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:46.520556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1043/2119] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:46.526063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ff057-01a6-421f-b5a3-73c03deaa36c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:46.542037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ff057-01a6-421f-b5a3-73c03deaa36c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:46.550410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ae1a7-d19d-4339-b5a9-646278a4aa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:46.551961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ae1a7-d19d-4339-b5a9-646278a4aa2e admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:46.552735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ae1a7-d19d-4339-b5a9-646278a4aa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:46.553446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ae1a7-d19d-4339-b5a9-646278a4aa2e admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:46.555333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ae1a7-d19d-4339-b5a9-646278a4aa2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:46.556332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ae1a7-d19d-4339-b5a9-646278a4aa2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:46.564660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ae1a7-d19d-4339-b5a9-646278a4aa2e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:46.593687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1077/2120] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:28:46.595694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3d2f6-0acf-4d29-a9ce-444a00eb34a7 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:46.615782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3d2f6-0acf-4d29-a9ce-444a00eb34a7 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:46.645586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4fb9f-7359-4f79-91d9-6d5247eabd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:46.650862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4fb9f-7359-4f79-91d9-6d5247eabd8b admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:46.651511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4fb9f-7359-4f79-91d9-6d5247eabd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:46.652122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4fb9f-7359-4f79-91d9-6d5247eabd8b admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:46.653019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4fb9f-7359-4f79-91d9-6d5247eabd8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:46.661133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4fb9f-7359-4f79-91d9-6d5247eabd8b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:46.802873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1044/2121] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:46 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:46.808321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ae1a7-d19d-4339-b5a9-646278a4aa2e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:46.827263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ae1a7-d19d-4339-b5a9-646278a4aa2e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:46.928716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6fd33-f64b-4092-bb2c-1e2b18804b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:46.929465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6fd33-f64b-4092-bb2c-1e2b18804b80 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:46.929888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6fd33-f64b-4092-bb2c-1e2b18804b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:46.930283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6fd33-f64b-4092-bb2c-1e2b18804b80 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:46.931314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6fd33-f64b-4092-bb2c-1e2b18804b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:46.934750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6fd33-f64b-4092-bb2c-1e2b18804b80 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:46.990240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4fb9f-7359-4f79-91d9-6d5247eabd8b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:46.992797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-87d4fb9f-7359-4f79-91d9-6d5247eabd8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:47.033056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4fb9f-7359-4f79-91d9-6d5247eabd8b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:47.040764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:47.041868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.042315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.042747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] PATH_INFO: `/v3/projects/465a192f9f2b4c95a4a6a3e9d85759c6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.044628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=465a192f9f2b4c95a4a6a3e9d85759c6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:47.048862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:47.053189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1078/2122] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:46 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:47.060377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6fd33-f64b-4092-bb2c-1e2b18804b80 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:47.077804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6fd33-f64b-4092-bb2c-1e2b18804b80 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:47.086404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c660ef-983f-47c3-a951-d058028c709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:47.087556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c660ef-983f-47c3-a951-d058028c709c admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.087934 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c660ef-983f-47c3-a951-d058028c709c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.088305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c660ef-983f-47c3-a951-d058028c709c admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.089192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c660ef-983f-47c3-a951-d058028c709c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:47.092140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c660ef-983f-47c3-a951-d058028c709c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:47.108405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1079/2123] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:47.113811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c660ef-983f-47c3-a951-d058028c709c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:47.125875 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c660ef-983f-47c3-a951-d058028c709c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:47.131585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7875a2-e99f-4ebc-947b-74edc8c6dae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:47.132219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7875a2-e99f-4ebc-947b-74edc8c6dae6 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.132633 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7875a2-e99f-4ebc-947b-74edc8c6dae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.132915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7875a2-e99f-4ebc-947b-74edc8c6dae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.133632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7875a2-e99f-4ebc-947b-74edc8c6dae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:47.135481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7875a2-e99f-4ebc-947b-74edc8c6dae6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:47.151100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1080/2124] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:47.155715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7875a2-e99f-4ebc-947b-74edc8c6dae6 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:47.167952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7875a2-e99f-4ebc-947b-74edc8c6dae6 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:47.173629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212477b-e0b8-4dfe-abe1-e210c6357b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:47.174318 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212477b-e0b8-4dfe-abe1-e210c6357b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.174664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212477b-e0b8-4dfe-abe1-e210c6357b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.174970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212477b-e0b8-4dfe-abe1-e210c6357b7c admin admin] PATH_INFO: `/v3/projects/664207f6e402484ba1a3eb5691cbe51c/users/1f2f13e8538f41f8b040c90fa74e2b58/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.197845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212477b-e0b8-4dfe-abe1-e210c6357b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=664207f6e402484ba1a3eb5691cbe51c, user_id=1f2f13e8538f41f8b040c90fa74e2b58, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:47.197845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212477b-e0b8-4dfe-abe1-e210c6357b7c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:47.207263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:47.211073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465a192f9f2b4c95a4a6a3e9d85759c6', 'request_id': 'req-470a7675-b74d-48ce-80f3-29555a50c619'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:47.221166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465a192f9f2b4c95a4a6a3e9d85759c6', 'request_id': 'req-470a7675-b74d-48ce-80f3-29555a50c619'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:47.292527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1081/2125] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:47 2022] PUT /identity/v3/projects/664207f6e402484ba1a3eb5691cbe51c/users/1f2f13e8538f41f8b040c90fa74e2b58/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:47.305531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465a192f9f2b4c95a4a6a3e9d85759c6', 'request_id': 'req-470a7675-b74d-48ce-80f3-29555a50c619'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:47.306451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0901-b798-40ce-a6c8-8bbc17bf09e5 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.306772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0901-b798-40ce-a6c8-8bbc17bf09e5 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.307068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0901-b798-40ce-a6c8-8bbc17bf09e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.329766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] The token cache is being invalidate because project 465a192f9f2b4c95a4a6a3e9d85759c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:47.330410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:47.332095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-470a7675-b74d-48ce-80f3-29555a50c619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:47.335150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1045/2126] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:46 2022] DELETE /identity/v3/projects/465a192f9f2b4c95a4a6a3e9d85759c6 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:47.343784 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc742f2-a54b-492e-a0f6-9e5276fd0a84 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.345335 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc742f2-a54b-492e-a0f6-9e5276fd0a84 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.345773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc742f2-a54b-492e-a0f6-9e5276fd0a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.348731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f4cf0901-b798-40ce-a6c8-8bbc17bf09e5 None None] MFA Rules not processed for user `1f2f13e8538f41f8b040c90fa74e2b58`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:47.350290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f4cf0901-b798-40ce-a6c8-8bbc17bf09e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:47.387271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-5bc742f2-a54b-492e-a0f6-9e5276fd0a84 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:47.389651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5bc742f2-a54b-492e-a0f6-9e5276fd0a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:47.494056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1082/2127] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:47.509466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0901-b798-40ce-a6c8-8bbc17bf09e5 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:47.511225 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f4cf0901-b798-40ce-a6c8-8bbc17bf09e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:47.517824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:47.525801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1046/2128] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:47.537350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881f5a6-97e5-425b-9fdb-2b90a7a444b4 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.537836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881f5a6-97e5-425b-9fdb-2b90a7a444b4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.538321 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881f5a6-97e5-425b-9fdb-2b90a7a444b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.572113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3881f5a6-97e5-425b-9fdb-2b90a7a444b4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:47.574062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3881f5a6-97e5-425b-9fdb-2b90a7a444b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:47.585617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1047/2129] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:47.590794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881f5a6-97e5-425b-9fdb-2b90a7a444b4 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:47.601816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881f5a6-97e5-425b-9fdb-2b90a7a444b4 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:47.606982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8a88c-10b0-4ee7-9c01-183710ebdfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:47.607697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8a88c-10b0-4ee7-9c01-183710ebdfe6 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.607851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8a88c-10b0-4ee7-9c01-183710ebdfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.608127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8a88c-10b0-4ee7-9c01-183710ebdfe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.608796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8a88c-10b0-4ee7-9c01-183710ebdfe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:47.609105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8a88c-10b0-4ee7-9c01-183710ebdfe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:47.610821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8a88c-10b0-4ee7-9c01-183710ebdfe6 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:47.624844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1048/2130] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:28:47.629661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8a88c-10b0-4ee7-9c01-183710ebdfe6 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:47.641025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8a88c-10b0-4ee7-9c01-183710ebdfe6 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:47.647448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83768c1-bde3-47e1-8ba1-7f8b5fc45e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:47.647924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0901-b798-40ce-a6c8-8bbc17bf09e5 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:47.648121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83768c1-bde3-47e1-8ba1-7f8b5fc45e20 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.648346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83768c1-bde3-47e1-8ba1-7f8b5fc45e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.648618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83768c1-bde3-47e1-8ba1-7f8b5fc45e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.649282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83768c1-bde3-47e1-8ba1-7f8b5fc45e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:47.651426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83768c1-bde3-47e1-8ba1-7f8b5fc45e20 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:47.654996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d351940-add8-4247-b3cb-8cc6d825f806 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:47.655831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351940-add8-4247-b3cb-8cc6d825f806 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.656356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351940-add8-4247-b3cb-8cc6d825f806 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.656769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351940-add8-4247-b3cb-8cc6d825f806 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.668495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d351940-add8-4247-b3cb-8cc6d825f806 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:47.671951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d351940-add8-4247-b3cb-8cc6d825f806 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:47.685278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1083/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:28:47.778694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1049/2132] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:47 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:47.786928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d351940-add8-4247-b3cb-8cc6d825f806 service nova] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:47.805505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d351940-add8-4247-b3cb-8cc6d825f806 service nova] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:47.919832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb373c7a-b0f5-46a8-aa71-dc2dd3cae4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:47.921106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb373c7a-b0f5-46a8-aa71-dc2dd3cae4ff admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:47.921584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb373c7a-b0f5-46a8-aa71-dc2dd3cae4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:47.922048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb373c7a-b0f5-46a8-aa71-dc2dd3cae4ff admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:47.923058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb373c7a-b0f5-46a8-aa71-dc2dd3cae4ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:47.925783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb373c7a-b0f5-46a8-aa71-dc2dd3cae4ff admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:48.046410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1084/2133] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:47 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:48.050127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83768c1-bde3-47e1-8ba1-7f8b5fc45e20 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:48.068277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83768c1-bde3-47e1-8ba1-7f8b5fc45e20 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:48.082762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc199b0d-35a2-46f1-a1a8-253fda526ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:48.083471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc199b0d-35a2-46f1-a1a8-253fda526ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:48.083794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc199b0d-35a2-46f1-a1a8-253fda526ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:48.084079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc199b0d-35a2-46f1-a1a8-253fda526ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:48.084724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc199b0d-35a2-46f1-a1a8-253fda526ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:48.086616 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc199b0d-35a2-46f1-a1a8-253fda526ff8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:48.107045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1050/2134] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:48.112210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb373c7a-b0f5-46a8-aa71-dc2dd3cae4ff admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:48.132480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb373c7a-b0f5-46a8-aa71-dc2dd3cae4ff admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:48.144386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92179fee-41bb-413d-8393-b1eae92774f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:48.144726 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92179fee-41bb-413d-8393-b1eae92774f3 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:48.145200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92179fee-41bb-413d-8393-b1eae92774f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:48.145704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92179fee-41bb-413d-8393-b1eae92774f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:48.146978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92179fee-41bb-413d-8393-b1eae92774f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:48.150265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92179fee-41bb-413d-8393-b1eae92774f3 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:48.171818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1085/2135] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:48.180191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc199b0d-35a2-46f1-a1a8-253fda526ff8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:48.196026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc199b0d-35a2-46f1-a1a8-253fda526ff8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:48.202538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a390e-062c-41dc-b74a-4a78cf3129f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:48.203669 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a390e-062c-41dc-b74a-4a78cf3129f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:48.204148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a390e-062c-41dc-b74a-4a78cf3129f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:48.204622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a390e-062c-41dc-b74a-4a78cf3129f6 admin admin] PATH_INFO: `/v3/projects/e75440603c194725989299f0928d1c40/users/254265aafb4c455ea472a1a4c4507ec2/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:48.231635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a390e-062c-41dc-b74a-4a78cf3129f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e75440603c194725989299f0928d1c40, user_id=254265aafb4c455ea472a1a4c4507ec2, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:48.235563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a390e-062c-41dc-b74a-4a78cf3129f6 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:48.356922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1051/2136] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:48 2022] PUT /identity/v3/projects/e75440603c194725989299f0928d1c40/users/254265aafb4c455ea472a1a4c4507ec2/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:48.371343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa6087-3f02-4dd5-9487-a91d5800c55c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:48.371739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa6087-3f02-4dd5-9487-a91d5800c55c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:48.371919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa6087-3f02-4dd5-9487-a91d5800c55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:48.412594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-42fa6087-3f02-4dd5-9487-a91d5800c55c None None] MFA Rules not processed for user `254265aafb4c455ea472a1a4c4507ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:48.414796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-42fa6087-3f02-4dd5-9487-a91d5800c55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:48.527108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:48.554894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1086/2137] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:48.600145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a390e-062c-41dc-b74a-4a78cf3129f6 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:48.602999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2c0a390e-062c-41dc-b74a-4a78cf3129f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:48.740092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a390e-062c-41dc-b74a-4a78cf3129f6 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:48.748208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdc0b2-ec37-4502-b6d9-3f95e9911463 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:48.749216 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdc0b2-ec37-4502-b6d9-3f95e9911463 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:48.749379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdc0b2-ec37-4502-b6d9-3f95e9911463 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:48.749550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdc0b2-ec37-4502-b6d9-3f95e9911463 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:48.776913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdc0b2-ec37-4502-b6d9-3f95e9911463 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:48.776913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdc0b2-ec37-4502-b6d9-3f95e9911463 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:48.785631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:48.808773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1052/2138] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:28:51.618735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa6087-3f02-4dd5-9487-a91d5800c55c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:51.618735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-42fa6087-3f02-4dd5-9487-a91d5800c55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:51.745291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa6087-3f02-4dd5-9487-a91d5800c55c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:51.752486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:51.754679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:51.755221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:51.755763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] PATH_INFO: `/v3/users/1f2f13e8538f41f8b040c90fa74e2b58` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:51.758051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2f13e8538f41f8b040c90fa74e2b58)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:51.761892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:52.029607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2f13e8538f41f8b040c90fa74e2b58'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:52.030886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2f13e8538f41f8b040c90fa74e2b58'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:52.032326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2f13e8538f41f8b040c90fa74e2b58'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:52.104861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2f13e8538f41f8b040c90fa74e2b58'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:52.116414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2f13e8538f41f8b040c90fa74e2b58'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:52.151062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1087/2139] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:51 2022] DELETE /identity/v3/users/1f2f13e8538f41f8b040c90fa74e2b58 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:52.157818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bf594-34f0-493c-a446-8f81bb78072f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:52.158527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bf594-34f0-493c-a446-8f81bb78072f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:52.158767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bf594-34f0-493c-a446-8f81bb78072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:52.192563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-9d6bf594-34f0-493c-a446-8f81bb78072f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:52.194152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9d6bf594-34f0-493c-a446-8f81bb78072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:52.268573 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1053/2140] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:52.280916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:52.282618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:52.370993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213047d-a667-4b2c-bd5a-40a27cd41026 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:52.380407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb926570-e2f2-4be8-aa76-791a7b9f9d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:52.381443 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb926570-e2f2-4be8-aa76-791a7b9f9d97 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:52.381882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb926570-e2f2-4be8-aa76-791a7b9f9d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:52.382338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb926570-e2f2-4be8-aa76-791a7b9f9d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:52.401127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb926570-e2f2-4be8-aa76-791a7b9f9d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:52.403948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb926570-e2f2-4be8-aa76-791a7b9f9d97 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:52.423076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1088/2141] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:28:53.052426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bf594-34f0-493c-a446-8f81bb78072f None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:53.054376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9d6bf594-34f0-493c-a446-8f81bb78072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:53.082344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bf594-34f0-493c-a446-8f81bb78072f None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:53.090463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:53.092407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:53.093306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:53.093974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] PATH_INFO: `/v3/projects/664207f6e402484ba1a3eb5691cbe51c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:53.097412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=664207f6e402484ba1a3eb5691cbe51c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:53.100901 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:53.275749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664207f6e402484ba1a3eb5691cbe51c', 'request_id': 'req-b5b4b221-e4c0-4274-a012-4bec3f031751'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:53.288348 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664207f6e402484ba1a3eb5691cbe51c', 'request_id': 'req-b5b4b221-e4c0-4274-a012-4bec3f031751'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:53.407767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664207f6e402484ba1a3eb5691cbe51c', 'request_id': 'req-b5b4b221-e4c0-4274-a012-4bec3f031751'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:53.423142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] The token cache is being invalidate because project 664207f6e402484ba1a3eb5691cbe51c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:53.423468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:53.424107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-b5b4b221-e4c0-4274-a012-4bec3f031751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:53.426287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1054/2142] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:53 2022] DELETE /identity/v3/projects/664207f6e402484ba1a3eb5691cbe51c => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:53.432338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2fc4a-76f7-4626-9ee5-fab061b2f394 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:53.433003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2fc4a-76f7-4626-9ee5-fab061b2f394 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:53.433003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2fc4a-76f7-4626-9ee5-fab061b2f394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:53.458851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6bf2fc4a-76f7-4626-9ee5-fab061b2f394 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:53.460283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6bf2fc4a-76f7-4626-9ee5-fab061b2f394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:53.494436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:53.564916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1089/2143] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:53.587146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4efeb-d3ea-4f98-9721-44fe80fe07a6 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:53.587146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4efeb-d3ea-4f98-9721-44fe80fe07a6 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:53.587146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4efeb-d3ea-4f98-9721-44fe80fe07a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:53.602447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-53f4efeb-d3ea-4f98-9721-44fe80fe07a6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:53.603738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-53f4efeb-d3ea-4f98-9721-44fe80fe07a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:53.612298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1055/2144] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:53.618526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2fc4a-76f7-4626-9ee5-fab061b2f394 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:53.628885 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2fc4a-76f7-4626-9ee5-fab061b2f394 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:53.634517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae6955-b04d-4fac-8b13-c7fdfd95b91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:53.635182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae6955-b04d-4fac-8b13-c7fdfd95b91a admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:53.635465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae6955-b04d-4fac-8b13-c7fdfd95b91a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:53.635739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae6955-b04d-4fac-8b13-c7fdfd95b91a admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:53.636422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae6955-b04d-4fac-8b13-c7fdfd95b91a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:53.636738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae6955-b04d-4fac-8b13-c7fdfd95b91a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:53.638688 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae6955-b04d-4fac-8b13-c7fdfd95b91a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:53.653263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1090/2145] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:28:53.658446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4efeb-d3ea-4f98-9721-44fe80fe07a6 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:53.664158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae6955-b04d-4fac-8b13-c7fdfd95b91a admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:53.667466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-65ae6955-b04d-4fac-8b13-c7fdfd95b91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:53.671984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4efeb-d3ea-4f98-9721-44fe80fe07a6 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:53.682875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53996186-c62a-483c-a24e-9a0f2f551086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:53.682875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53996186-c62a-483c-a24e-9a0f2f551086 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:53.682875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53996186-c62a-483c-a24e-9a0f2f551086 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:53.682875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53996186-c62a-483c-a24e-9a0f2f551086 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:53.683499 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53996186-c62a-483c-a24e-9a0f2f551086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:53.684284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae6955-b04d-4fac-8b13-c7fdfd95b91a admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:53.685109 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53996186-c62a-483c-a24e-9a0f2f551086 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:53.694379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:53.695536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:53.696051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:53.696877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] PATH_INFO: `/v3/users/710633b814b14e23a13a142aa12fc797` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:53.699989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=710633b814b14e23a13a142aa12fc797)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:53.704135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:53.839418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1056/2146] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:53 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:53.844284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53996186-c62a-483c-a24e-9a0f2f551086 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:53.918782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:53.926032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53996186-c62a-483c-a24e-9a0f2f551086 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:53.931428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98705526-16eb-4b6a-9b76-77237c325bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:53.932076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98705526-16eb-4b6a-9b76-77237c325bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:53.932356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98705526-16eb-4b6a-9b76-77237c325bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:53.932631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98705526-16eb-4b6a-9b76-77237c325bd6 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:53.933285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98705526-16eb-4b6a-9b76-77237c325bd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:53.935334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98705526-16eb-4b6a-9b76-77237c325bd6 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:53.992304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '710633b814b14e23a13a142aa12fc797'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:53.993345 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '710633b814b14e23a13a142aa12fc797'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:53.994108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '710633b814b14e23a13a142aa12fc797'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:54.041029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1057/2147] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:53 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:54.043879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98705526-16eb-4b6a-9b76-77237c325bd6 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:54.045844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-98705526-16eb-4b6a-9b76-77237c325bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:54.060504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98705526-16eb-4b6a-9b76-77237c325bd6 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:54.066425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b4a58-ffce-4638-afa2-fda2adf29260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:54.067446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b4a58-ffce-4638-afa2-fda2adf29260 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:54.067759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b4a58-ffce-4638-afa2-fda2adf29260 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:54.068125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b4a58-ffce-4638-afa2-fda2adf29260 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:54.068951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b4a58-ffce-4638-afa2-fda2adf29260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:54.070884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b4a58-ffce-4638-afa2-fda2adf29260 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:54.083519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1058/2148] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:54.088093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '710633b814b14e23a13a142aa12fc797'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:54.088509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b4a58-ffce-4638-afa2-fda2adf29260 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:54.102302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b4a58-ffce-4638-afa2-fda2adf29260 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:54.103238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-bc61dfd4-aa96-442e-bc7f-1dc887fa1454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '710633b814b14e23a13a142aa12fc797'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:54.107866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a52394-ce3b-4bcb-97b9-be2ebe46ec93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:54.108496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a52394-ce3b-4bcb-97b9-be2ebe46ec93 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:54.108834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a52394-ce3b-4bcb-97b9-be2ebe46ec93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:54.109113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a52394-ce3b-4bcb-97b9-be2ebe46ec93 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:54.109871 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a52394-ce3b-4bcb-97b9-be2ebe46ec93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:54.111594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a52394-ce3b-4bcb-97b9-be2ebe46ec93 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:54.124649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1059/2149] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:54.129201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a52394-ce3b-4bcb-97b9-be2ebe46ec93 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:54.132733 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1091/2150] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:28:53 2022] DELETE /identity/v3/users/710633b814b14e23a13a142aa12fc797 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:54.140304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a52394-ce3b-4bcb-97b9-be2ebe46ec93 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:54.144128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbca55a-fae1-4594-b1eb-915ff9be8511 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:54.144636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbca55a-fae1-4594-b1eb-915ff9be8511 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:54.145146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbca55a-fae1-4594-b1eb-915ff9be8511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:54.181013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-bbbca55a-fae1-4594-b1eb-915ff9be8511 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:54.183306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bbbca55a-fae1-4594-b1eb-915ff9be8511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:54.216301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a7f4a-a807-46c4-8df7-220afd583e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:54.217451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a7f4a-a807-46c4-8df7-220afd583e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:54.217926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a7f4a-a807-46c4-8df7-220afd583e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:54.218421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a7f4a-a807-46c4-8df7-220afd583e6f admin admin] PATH_INFO: `/v3/projects/e9ecdd9e74e0435c8bd03065b5225403/users/371d90bbb0e0461aafc024284acf7590/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:54.243490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a7f4a-a807-46c4-8df7-220afd583e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ecdd9e74e0435c8bd03065b5225403, user_id=371d90bbb0e0461aafc024284acf7590, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:54.246921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a7f4a-a807-46c4-8df7-220afd583e6f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:54.279119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1092/2151] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:54.293940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbca55a-fae1-4594-b1eb-915ff9be8511 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:54.295446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bbbca55a-fae1-4594-b1eb-915ff9be8511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:54.327002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:54.364313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1060/2152] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:54 2022] PUT /identity/v3/projects/e9ecdd9e74e0435c8bd03065b5225403/users/371d90bbb0e0461aafc024284acf7590/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:54.370720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec46e53-d259-4154-b1a9-87cc7606ef82 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:54.371180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec46e53-d259-4154-b1a9-87cc7606ef82 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:54.371677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec46e53-d259-4154-b1a9-87cc7606ef82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:54.398872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbca55a-fae1-4594-b1eb-915ff9be8511 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:54.417932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2ec46e53-d259-4154-b1a9-87cc7606ef82 None None] MFA Rules not processed for user `371d90bbb0e0461aafc024284acf7590`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:54.419654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2ec46e53-d259-4154-b1a9-87cc7606ef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:54.477514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ee335-8e8a-4f27-b9d5-e9b9d6d81408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:54.478244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ee335-8e8a-4f27-b9d5-e9b9d6d81408 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:54.478618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ee335-8e8a-4f27-b9d5-e9b9d6d81408 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:54.479033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ee335-8e8a-4f27-b9d5-e9b9d6d81408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:54.554208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1061/2153] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:28:54.556170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec46e53-d259-4154-b1a9-87cc7606ef82 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:54.627359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ee335-8e8a-4f27-b9d5-e9b9d6d81408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:54.631074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ee335-8e8a-4f27-b9d5-e9b9d6d81408 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:54.651528 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1093/2154] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:28:54.651796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:54.693870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec46e53-d259-4154-b1a9-87cc7606ef82 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:54.699191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0962fa3-1a34-44d9-a5c1-5487c1aed1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:54.699961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0962fa3-1a34-44d9-a5c1-5487c1aed1d6 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:54.700434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0962fa3-1a34-44d9-a5c1-5487c1aed1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:54.700585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0962fa3-1a34-44d9-a5c1-5487c1aed1d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:54.701229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0962fa3-1a34-44d9-a5c1-5487c1aed1d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:54.703751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0962fa3-1a34-44d9-a5c1-5487c1aed1d6 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:54.848365 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1062/2155] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:54 2022] POST /identity/v3/projects => generated 371 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:54.853049 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ee335-8e8a-4f27-b9d5-e9b9d6d81408 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:54.865051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ee335-8e8a-4f27-b9d5-e9b9d6d81408 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:54.939089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979f1ae-8737-43bc-b9b1-fe1c7b4a8ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:54.939718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979f1ae-8737-43bc-b9b1-fe1c7b4a8ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:54.939993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979f1ae-8737-43bc-b9b1-fe1c7b4a8ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:54.940266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979f1ae-8737-43bc-b9b1-fe1c7b4a8ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:54.941877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979f1ae-8737-43bc-b9b1-fe1c7b4a8ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:54.944237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979f1ae-8737-43bc-b9b1-fe1c7b4a8ad8 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:55.049192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1094/2156] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:54 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:55.055275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0962fa3-1a34-44d9-a5c1-5487c1aed1d6 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:55.072071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0962fa3-1a34-44d9-a5c1-5487c1aed1d6 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:55.081183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e44d13-b4e3-4665-bbd2-54adc7f145e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:55.082284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44d13-b4e3-4665-bbd2-54adc7f145e1 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.082827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44d13-b4e3-4665-bbd2-54adc7f145e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.083305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44d13-b4e3-4665-bbd2-54adc7f145e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.084560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e44d13-b4e3-4665-bbd2-54adc7f145e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:55.088117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e44d13-b4e3-4665-bbd2-54adc7f145e1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:55.115579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1063/2157] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:55.117630 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979f1ae-8737-43bc-b9b1-fe1c7b4a8ad8 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:55.137259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979f1ae-8737-43bc-b9b1-fe1c7b4a8ad8 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:55.145735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a413d0-e6fe-423c-9371-7c6723fabed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:55.147062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a413d0-e6fe-423c-9371-7c6723fabed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.147992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a413d0-e6fe-423c-9371-7c6723fabed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.148614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a413d0-e6fe-423c-9371-7c6723fabed2 admin admin] PATH_INFO: `/v3/projects/6ab980c9c48349a7ab98d58ec3c57196/users/7192defda870422b9e5875bb63bbbe4f/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.170146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:55.177371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a413d0-e6fe-423c-9371-7c6723fabed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ab980c9c48349a7ab98d58ec3c57196, user_id=7192defda870422b9e5875bb63bbbe4f, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:55.183311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a413d0-e6fe-423c-9371-7c6723fabed2 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:55.246392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e44d13-b4e3-4665-bbd2-54adc7f145e1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:55.249774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-91e44d13-b4e3-4665-bbd2-54adc7f145e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:55.276267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e44d13-b4e3-4665-bbd2-54adc7f145e1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:55.281134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1095/2158] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:55 2022] PUT /identity/v3/projects/6ab980c9c48349a7ab98d58ec3c57196/users/7192defda870422b9e5875bb63bbbe4f/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:55.289586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a413d0-e6fe-423c-9371-7c6723fabed2 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:55.380795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a413d0-e6fe-423c-9371-7c6723fabed2 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:55.384440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:55.385552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.386019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.386681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] PATH_INFO: `/v3/projects/4de02c4fd1fb48d589ab31a1eadc2cd6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.386948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa3c1f-ce3b-4a81-b48d-52bfba5d429b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:55.387685 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa3c1f-ce3b-4a81-b48d-52bfba5d429b admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.388070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa3c1f-ce3b-4a81-b48d-52bfba5d429b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.388456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa3c1f-ce3b-4a81-b48d-52bfba5d429b admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.389051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de02c4fd1fb48d589ab31a1eadc2cd6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:55.389583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa3c1f-ce3b-4a81-b48d-52bfba5d429b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:55.391575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa3c1f-ce3b-4a81-b48d-52bfba5d429b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:55.392845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:55.407915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1096/2159] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:28:55.409752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa3c1f-ce3b-4a81-b48d-52bfba5d429b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:55.421165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa3c1f-ce3b-4a81-b48d-52bfba5d429b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:55.426540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65e670-1167-4761-b8ea-d71fcf2f05c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:55.427252 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65e670-1167-4761-b8ea-d71fcf2f05c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.427537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65e670-1167-4761-b8ea-d71fcf2f05c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.427940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65e670-1167-4761-b8ea-d71fcf2f05c1 admin admin] PATH_INFO: `/v3/projects/6ab980c9c48349a7ab98d58ec3c57196/users/7192defda870422b9e5875bb63bbbe4f/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.429836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65e670-1167-4761-b8ea-d71fcf2f05c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ab980c9c48349a7ab98d58ec3c57196, user_id=7192defda870422b9e5875bb63bbbe4f, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:55.432655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65e670-1167-4761-b8ea-d71fcf2f05c1 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:55.448244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1097/2160] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:55 2022] PUT /identity/v3/projects/6ab980c9c48349a7ab98d58ec3c57196/users/7192defda870422b9e5875bb63bbbe4f/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:55.468392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36992fab-0e02-4a9e-b844-b25966610f84 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.468756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36992fab-0e02-4a9e-b844-b25966610f84 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.469101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36992fab-0e02-4a9e-b844-b25966610f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.504295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-36992fab-0e02-4a9e-b844-b25966610f84 None None] MFA Rules not processed for user `7192defda870422b9e5875bb63bbbe4f`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:55.505987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-36992fab-0e02-4a9e-b844-b25966610f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:55.539666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de02c4fd1fb48d589ab31a1eadc2cd6', 'request_id': 'req-c6148e79-12e1-434a-8681-e14da239aa8c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:55.553006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de02c4fd1fb48d589ab31a1eadc2cd6', 'request_id': 'req-c6148e79-12e1-434a-8681-e14da239aa8c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:55.624152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1098/2161] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:55.631567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de02c4fd1fb48d589ab31a1eadc2cd6', 'request_id': 'req-c6148e79-12e1-434a-8681-e14da239aa8c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:55.643085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36992fab-0e02-4a9e-b844-b25966610f84 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:55.645461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-36992fab-0e02-4a9e-b844-b25966610f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:55.650001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] The token cache is being invalidate because project 4de02c4fd1fb48d589ab31a1eadc2cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:28:55.650302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:55.651128 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c6148e79-12e1-434a-8681-e14da239aa8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:28:55.658235 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1064/2162] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:28:55 2022] DELETE /identity/v3/projects/4de02c4fd1fb48d589ab31a1eadc2cd6 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:28:55.661126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c1956-d5d8-4a85-9df5-a0f9c59a20ab None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.661508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c1956-d5d8-4a85-9df5-a0f9c59a20ab None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.662374 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c1956-d5d8-4a85-9df5-a0f9c59a20ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.693711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2b6c1956-d5d8-4a85-9df5-a0f9c59a20ab None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:55.695335 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2b6c1956-d5d8-4a85-9df5-a0f9c59a20ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:55.750913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:55.783437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36992fab-0e02-4a9e-b844-b25966610f84 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:55.790685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1065/2163] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:55.798050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:55.799221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1312d-7db3-4483-98a3-6189501151ad None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.799705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1312d-7db3-4483-98a3-6189501151ad None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.800556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1312d-7db3-4483-98a3-6189501151ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.824603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-f9d1312d-7db3-4483-98a3-6189501151ad None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:55.826222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f9d1312d-7db3-4483-98a3-6189501151ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:55.836263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1066/2164] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:55.841831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1312d-7db3-4483-98a3-6189501151ad None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:55.859565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1312d-7db3-4483-98a3-6189501151ad None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:55.863511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3bab1-7699-452f-9804-bb8ec2107fe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:55.863511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3bab1-7699-452f-9804-bb8ec2107fe8 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.863511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3bab1-7699-452f-9804-bb8ec2107fe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.863511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3bab1-7699-452f-9804-bb8ec2107fe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.865447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-eef3bab1-7699-452f-9804-bb8ec2107fe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:55.866354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735f96f-90f4-4dac-80f2-f472e43086ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:55.867332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735f96f-90f4-4dac-80f2-f472e43086ef admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.867793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735f96f-90f4-4dac-80f2-f472e43086ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.868275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735f96f-90f4-4dac-80f2-f472e43086ef admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.869181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735f96f-90f4-4dac-80f2-f472e43086ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:28:55.869653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735f96f-90f4-4dac-80f2-f472e43086ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:55.871500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735f96f-90f4-4dac-80f2-f472e43086ef admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:55.887189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1067/2165] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:28:55.892306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735f96f-90f4-4dac-80f2-f472e43086ef admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:55.903670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735f96f-90f4-4dac-80f2-f472e43086ef admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:55.909166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7016-3740-4e51-9323-5895f2478a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:55.910455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7016-3740-4e51-9323-5895f2478a95 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:55.910841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7016-3740-4e51-9323-5895f2478a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:55.911209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7016-3740-4e51-9323-5895f2478a95 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:55.911881 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a7016-3740-4e51-9323-5895f2478a95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:55.913971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a7016-3740-4e51-9323-5895f2478a95 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:55.980732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3bab1-7699-452f-9804-bb8ec2107fe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:55.982747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3bab1-7699-452f-9804-bb8ec2107fe8 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:55.994218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1099/2166] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:28:56.028648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1068/2167] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:28:55 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:28:56.032954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3bab1-7699-452f-9804-bb8ec2107fe8 service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:56.045264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3bab1-7699-452f-9804-bb8ec2107fe8 service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:56.117236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc44e27-3b8a-4468-9ec2-966063acb483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:56.118055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44e27-3b8a-4468-9ec2-966063acb483 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:56.118195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44e27-3b8a-4468-9ec2-966063acb483 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:56.118380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44e27-3b8a-4468-9ec2-966063acb483 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:56.119055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc44e27-3b8a-4468-9ec2-966063acb483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:56.121067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc44e27-3b8a-4468-9ec2-966063acb483 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:56.216900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7016-3740-4e51-9323-5895f2478a95 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:56.218823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1100/2168] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:28:56 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:28:56.219627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3d6a7016-3740-4e51-9323-5895f2478a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:56.225554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc44e27-3b8a-4468-9ec2-966063acb483 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:56.237231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc44e27-3b8a-4468-9ec2-966063acb483 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:56.243395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5b377-2c00-422c-aebf-6085dd7c289c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:56.244026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5b377-2c00-422c-aebf-6085dd7c289c admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:56.244352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5b377-2c00-422c-aebf-6085dd7c289c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:56.244688 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5b377-2c00-422c-aebf-6085dd7c289c admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:56.245402 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5b377-2c00-422c-aebf-6085dd7c289c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:56.247151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5b377-2c00-422c-aebf-6085dd7c289c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:56.259229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1101/2169] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:56.263653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5b377-2c00-422c-aebf-6085dd7c289c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:56.274940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5b377-2c00-422c-aebf-6085dd7c289c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:56.280084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e3a91-f12e-4b42-86b6-8b8284fb06a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:56.280704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e3a91-f12e-4b42-86b6-8b8284fb06a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:56.280978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e3a91-f12e-4b42-86b6-8b8284fb06a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:56.281265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e3a91-f12e-4b42-86b6-8b8284fb06a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:56.281899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e3a91-f12e-4b42-86b6-8b8284fb06a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:56.283582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e3a91-f12e-4b42-86b6-8b8284fb06a1 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:56.303588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1102/2170] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:28:56.305054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e3a91-f12e-4b42-86b6-8b8284fb06a1 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:56.317888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e3a91-f12e-4b42-86b6-8b8284fb06a1 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:56.323146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77b3b6-e06b-45f4-9553-8e878cfcee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:56.323839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77b3b6-e06b-45f4-9553-8e878cfcee79 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:56.324106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77b3b6-e06b-45f4-9553-8e878cfcee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:56.324374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77b3b6-e06b-45f4-9553-8e878cfcee79 admin admin] PATH_INFO: `/v3/projects/bdaa04b86d854028bae8ad5212aa00f7/users/dc9a9e1a926743c691d09b3f775b667e/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:56.328033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7016-3740-4e51-9323-5895f2478a95 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:56.332725 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce7993-8256-410a-a50f-1f7fa57662ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:56.333361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7993-8256-410a-a50f-1f7fa57662ef service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:56.333643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7993-8256-410a-a50f-1f7fa57662ef service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:56.333892 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7993-8256-410a-a50f-1f7fa57662ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:56.335974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-05ce7993-8256-410a-a50f-1f7fa57662ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:56.345302 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77b3b6-e06b-45f4-9553-8e878cfcee79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdaa04b86d854028bae8ad5212aa00f7, user_id=dc9a9e1a926743c691d09b3f775b667e, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:56.349723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77b3b6-e06b-45f4-9553-8e878cfcee79 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:56.416608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce7993-8256-410a-a50f-1f7fa57662ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:56.418855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce7993-8256-410a-a50f-1f7fa57662ef service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:56.427674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1103/2171] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:28:56 2022] PUT /identity/v3/projects/bdaa04b86d854028bae8ad5212aa00f7/users/dc9a9e1a926743c691d09b3f775b667e/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:28:56.432936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1069/2172] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:28:56.439440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3b744-7499-4bd7-be41-ca1a1a129bed None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:56.439991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3b744-7499-4bd7-be41-ca1a1a129bed None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:56.439991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3b744-7499-4bd7-be41-ca1a1a129bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:56.454906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:56.479731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-88e3b744-7499-4bd7-be41-ca1a1a129bed None None] MFA Rules not processed for user `dc9a9e1a926743c691d09b3f775b667e`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:28:56.481756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-88e3b744-7499-4bd7-be41-ca1a1a129bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:56.575286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1104/2173] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:28:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:28:56.590083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce7993-8256-410a-a50f-1f7fa57662ef service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:28:56.592554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-05ce7993-8256-410a-a50f-1f7fa57662ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:28:56.609553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce7993-8256-410a-a50f-1f7fa57662ef service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:28:56.678349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd28c76-8f0c-4032-97a1-443c15699ebe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:28:56.679187 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd28c76-8f0c-4032-97a1-443c15699ebe service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:28:56.679564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd28c76-8f0c-4032-97a1-443c15699ebe service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:28:56.680000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd28c76-8f0c-4032-97a1-443c15699ebe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:28:56.689717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:28:56.693307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd28c76-8f0c-4032-97a1-443c15699ebe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:28:56.694976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd28c76-8f0c-4032-97a1-443c15699ebe service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:28:56.706852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1070/2174] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:28:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:29:06.571621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=29 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:29:06.830537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:29:10.980113 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3b744-7499-4bd7-be41-ca1a1a129bed None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:11.088208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3b744-7499-4bd7-be41-ca1a1a129bed None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:11.093256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:11.093918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:11.094197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:11.094459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] PATH_INFO: `/v3/users/dc9a9e1a926743c691d09b3f775b667e` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:11.095736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc9a9e1a926743c691d09b3f775b667e)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:11.099345 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:11.376146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc9a9e1a926743c691d09b3f775b667e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:11.377144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc9a9e1a926743c691d09b3f775b667e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:11.378463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc9a9e1a926743c691d09b3f775b667e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:11.494426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc9a9e1a926743c691d09b3f775b667e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:11.506557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc9a9e1a926743c691d09b3f775b667e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:11.544757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1105/2175] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:29:10 2022] DELETE /identity/v3/users/dc9a9e1a926743c691d09b3f775b667e => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:29:11.550540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc414daa-dc6f-4b7e-83a5-b07a7e785b85 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:11.550540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc414daa-dc6f-4b7e-83a5-b07a7e785b85 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:11.550540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc414daa-dc6f-4b7e-83a5-b07a7e785b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:11.583068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-fc414daa-dc6f-4b7e-83a5-b07a7e785b85 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:29:11.584877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fc414daa-dc6f-4b7e-83a5-b07a7e785b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:11.696986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1071/2176] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:29:11.713094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:11.714882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:11.802611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d76aae-f905-422e-9cd8-b4baca659065 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:11.809038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfcaa6-9174-4a35-91e4-173c8ea1779d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:11.809686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfcaa6-9174-4a35-91e4-173c8ea1779d service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:11.809970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfcaa6-9174-4a35-91e4-173c8ea1779d service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:11.810243 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfcaa6-9174-4a35-91e4-173c8ea1779d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:11.822213 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfcaa6-9174-4a35-91e4-173c8ea1779d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:11.823839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfcaa6-9174-4a35-91e4-173c8ea1779d service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:11.837192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1106/2177] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:29:12.480045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc414daa-dc6f-4b7e-83a5-b07a7e785b85 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:12.482277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fc414daa-dc6f-4b7e-83a5-b07a7e785b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:12.506688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc414daa-dc6f-4b7e-83a5-b07a7e785b85 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:12.515047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:12.516916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:12.516916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:12.516916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] PATH_INFO: `/v3/projects/bdaa04b86d854028bae8ad5212aa00f7` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:12.519334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdaa04b86d854028bae8ad5212aa00f7)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:12.521748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:12.652646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdaa04b86d854028bae8ad5212aa00f7', 'request_id': 'req-ac73eb16-656e-47b6-af02-f58cb9401291'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:12.662891 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdaa04b86d854028bae8ad5212aa00f7', 'request_id': 'req-ac73eb16-656e-47b6-af02-f58cb9401291'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:12.734536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdaa04b86d854028bae8ad5212aa00f7', 'request_id': 'req-ac73eb16-656e-47b6-af02-f58cb9401291'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:12.751394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] The token cache is being invalidate because project bdaa04b86d854028bae8ad5212aa00f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:29:12.751923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:12.753620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ac73eb16-656e-47b6-af02-f58cb9401291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:12.756732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1072/2178] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:29:12 2022] DELETE /identity/v3/projects/bdaa04b86d854028bae8ad5212aa00f7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:29:12.764891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba490b3e-dbff-483e-8411-171ee4255dfc None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:12.765703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba490b3e-dbff-483e-8411-171ee4255dfc None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:12.765703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba490b3e-dbff-483e-8411-171ee4255dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:12.792842 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-ba490b3e-dbff-483e-8411-171ee4255dfc None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:29:12.794360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ba490b3e-dbff-483e-8411-171ee4255dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:12.821193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:29:12.897956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1107/2179] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:29:12.900737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ade749-3a0f-4dc4-9771-8208a7372a13 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:12.901735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ade749-3a0f-4dc4-9771-8208a7372a13 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:12.901889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ade749-3a0f-4dc4-9771-8208a7372a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:12.932039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-03ade749-3a0f-4dc4-9771-8208a7372a13 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:29:12.933534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-03ade749-3a0f-4dc4-9771-8208a7372a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:12.942084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1073/2180] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:29:12.946826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba490b3e-dbff-483e-8411-171ee4255dfc None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:12.958382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba490b3e-dbff-483e-8411-171ee4255dfc None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:12.963628 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1087759-895d-43f9-83c7-16f7a999b893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:12.964267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1087759-895d-43f9-83c7-16f7a999b893 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:12.964549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1087759-895d-43f9-83c7-16f7a999b893 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:12.964820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1087759-895d-43f9-83c7-16f7a999b893 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:12.965597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1087759-895d-43f9-83c7-16f7a999b893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:29:12.965800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1087759-895d-43f9-83c7-16f7a999b893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:12.967505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1087759-895d-43f9-83c7-16f7a999b893 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:12.982761 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1108/2181] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:29:12.988115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ade749-3a0f-4dc4-9771-8208a7372a13 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:13.013532 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ade749-3a0f-4dc4-9771-8208a7372a13 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:13.016063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93b12d-99f7-4498-86d7-89033fe38ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:13.016801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93b12d-99f7-4498-86d7-89033fe38ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:13.017644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93b12d-99f7-4498-86d7-89033fe38ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:13.018057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93b12d-99f7-4498-86d7-89033fe38ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:13.019643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93b12d-99f7-4498-86d7-89033fe38ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:13.022348 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93b12d-99f7-4498-86d7-89033fe38ff4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:13.195181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1074/2182] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:29:12 2022] POST /identity/v3/projects => generated 369 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:29:13.196709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1087759-895d-43f9-83c7-16f7a999b893 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:13.212626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1087759-895d-43f9-83c7-16f7a999b893 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:13.300103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef1ec6-a669-4ce0-9206-23d5dd52608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:13.300737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef1ec6-a669-4ce0-9206-23d5dd52608f admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:13.301018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef1ec6-a669-4ce0-9206-23d5dd52608f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:13.301289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef1ec6-a669-4ce0-9206-23d5dd52608f admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:13.302731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef1ec6-a669-4ce0-9206-23d5dd52608f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:13.303983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef1ec6-a669-4ce0-9206-23d5dd52608f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:13.391500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1109/2183] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:29:13 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:29:13.393187 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93b12d-99f7-4498-86d7-89033fe38ff4 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:13.407556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93b12d-99f7-4498-86d7-89033fe38ff4 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:13.414447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef6ef3-81aa-497c-8e27-8ce0f84034b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:13.415512 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef6ef3-81aa-497c-8e27-8ce0f84034b4 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:13.415997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef6ef3-81aa-497c-8e27-8ce0f84034b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:13.416482 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef6ef3-81aa-497c-8e27-8ce0f84034b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:13.417668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef6ef3-81aa-497c-8e27-8ce0f84034b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:13.421149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef6ef3-81aa-497c-8e27-8ce0f84034b4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:13.445944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1075/2184] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:29:13.459710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef1ec6-a669-4ce0-9206-23d5dd52608f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:13.471486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef1ec6-a669-4ce0-9206-23d5dd52608f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:13.477181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cacb04-e3f0-4ccb-b2a7-53c2520d936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:13.478084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cacb04-e3f0-4ccb-b2a7-53c2520d936f admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:13.478357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cacb04-e3f0-4ccb-b2a7-53c2520d936f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:13.478702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cacb04-e3f0-4ccb-b2a7-53c2520d936f admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:13.479374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cacb04-e3f0-4ccb-b2a7-53c2520d936f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:13.481399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cacb04-e3f0-4ccb-b2a7-53c2520d936f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:13.507203 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1110/2185] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:29:13.512832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef6ef3-81aa-497c-8e27-8ce0f84034b4 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:13.531418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef6ef3-81aa-497c-8e27-8ce0f84034b4 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:13.539424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c8593-7a32-4fe0-a14b-a09a69fb7800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:13.540510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c8593-7a32-4fe0-a14b-a09a69fb7800 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:13.540946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c8593-7a32-4fe0-a14b-a09a69fb7800 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:13.541345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c8593-7a32-4fe0-a14b-a09a69fb7800 admin admin] PATH_INFO: `/v3/projects/279398a188ca4c7b8c8b502664aac429/users/9478834ef55643d79abf01a7cd23ed71/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:13.569579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c8593-7a32-4fe0-a14b-a09a69fb7800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=279398a188ca4c7b8c8b502664aac429, user_id=9478834ef55643d79abf01a7cd23ed71, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:13.575063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c8593-7a32-4fe0-a14b-a09a69fb7800 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:13.654151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1076/2186] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:29:13 2022] PUT /identity/v3/projects/279398a188ca4c7b8c8b502664aac429/users/9478834ef55643d79abf01a7cd23ed71/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:29:13.667722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc93189-2d41-4913-b39d-c6648f07af28 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:13.668005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc93189-2d41-4913-b39d-c6648f07af28 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:13.668278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc93189-2d41-4913-b39d-c6648f07af28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:13.702356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-5dc93189-2d41-4913-b39d-c6648f07af28 None None] MFA Rules not processed for user `9478834ef55643d79abf01a7cd23ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:29:13.703641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5dc93189-2d41-4913-b39d-c6648f07af28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:13.785891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1111/2187] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:29:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:29:13.799527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c8593-7a32-4fe0-a14b-a09a69fb7800 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:13.802447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bb7c8593-7a32-4fe0-a14b-a09a69fb7800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:13.844577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:29:13.900880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c8593-7a32-4fe0-a14b-a09a69fb7800 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:13.904923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c429d-420f-48e4-8faf-40ea01213cf0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea4c2870e2754f3188228fc441950df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:13.905558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c429d-420f-48e4-8faf-40ea01213cf0 service glance] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:13.905864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c429d-420f-48e4-8faf-40ea01213cf0 service glance] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:13.906135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c429d-420f-48e4-8faf-40ea01213cf0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:13.919042 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c429d-420f-48e4-8faf-40ea01213cf0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:13.920907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c429d-420f-48e4-8faf-40ea01213cf0 service glance] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:13.932117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1077/2188] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:29:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:29:23.780555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:29:23.944616 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=17 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:29:25.564108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc93189-2d41-4913-b39d-c6648f07af28 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:25.567435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5dc93189-2d41-4913-b39d-c6648f07af28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:25.700422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc93189-2d41-4913-b39d-c6648f07af28 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:25.706978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:25.707929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:25.708425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:25.708910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] PATH_INFO: `/v3/users/03a65de47d5d41ab858191a32058bcdb` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:25.713128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a65de47d5d41ab858191a32058bcdb)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:25.715254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:25.996843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a65de47d5d41ab858191a32058bcdb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:25.997920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a65de47d5d41ab858191a32058bcdb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:25.999288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a65de47d5d41ab858191a32058bcdb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:26.076109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a65de47d5d41ab858191a32058bcdb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:26.086366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a65de47d5d41ab858191a32058bcdb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:26.113297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1112/2189] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:29:25 2022] DELETE /identity/v3/users/03a65de47d5d41ab858191a32058bcdb => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:29:26.114817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564537c-c198-4519-b519-8983fa7cdd76 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:26.115575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564537c-c198-4519-b519-8983fa7cdd76 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:26.115741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564537c-c198-4519-b519-8983fa7cdd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:26.137600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1564537c-c198-4519-b519-8983fa7cdd76 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:29:26.142253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1564537c-c198-4519-b519-8983fa7cdd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:26.216388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1078/2190] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:29:26.225384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:26.227267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:26.297187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1dfe86-ea9b-4011-96a7-39dbf3774c5a admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:26.302156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47876b-c7c7-44bd-9b78-cf23622ee522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:26.303043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47876b-c7c7-44bd-9b78-cf23622ee522 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:26.303321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47876b-c7c7-44bd-9b78-cf23622ee522 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:26.303595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47876b-c7c7-44bd-9b78-cf23622ee522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:26.314383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47876b-c7c7-44bd-9b78-cf23622ee522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:26.316001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47876b-c7c7-44bd-9b78-cf23622ee522 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:26.327131 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1113/2191] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:29:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:29:27.089340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564537c-c198-4519-b519-8983fa7cdd76 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:27.093155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1564537c-c198-4519-b519-8983fa7cdd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:27.133575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564537c-c198-4519-b519-8983fa7cdd76 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:27.143473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:27.143473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:27.143473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:27.143473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] PATH_INFO: `/v3/projects/84ff1f60d005454a90e647f1ace0cf43` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:27.144062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ff1f60d005454a90e647f1ace0cf43)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:27.144980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:27.320019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ff1f60d005454a90e647f1ace0cf43', 'request_id': 'req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:27.334688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ff1f60d005454a90e647f1ace0cf43', 'request_id': 'req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:27.413523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ff1f60d005454a90e647f1ace0cf43', 'request_id': 'req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:27.435966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] The token cache is being invalidate because project 84ff1f60d005454a90e647f1ace0cf43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:29:27.436617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:27.437529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:27.440807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1079/2192] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:29:27 2022] DELETE /identity/v3/projects/84ff1f60d005454a90e647f1ace0cf43 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:29:30.192817 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47876b-c7c7-44bd-9b78-cf23622ee522 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:30.194902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0d47876b-c7c7-44bd-9b78-cf23622ee522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:30.225318 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:29:30.278393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47876b-c7c7-44bd-9b78-cf23622ee522 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:30.385397 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9982f0-aef6-4ba1-9cde-894f24f71e61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:30.386436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9982f0-aef6-4ba1-9cde-894f24f71e61 service placement] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:30.386907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9982f0-aef6-4ba1-9cde-894f24f71e61 service placement] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:30.387464 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9982f0-aef6-4ba1-9cde-894f24f71e61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:30.390721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5f9982f0-aef6-4ba1-9cde-894f24f71e61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:30.412286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9982f0-aef6-4ba1-9cde-894f24f71e61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:30.413698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9982f0-aef6-4ba1-9cde-894f24f71e61 service placement] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:30.425959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1114/2193] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:29:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:29:31.154455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:31.157068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:31.247390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20faa7c-c76b-44e7-8f31-1463e18ac9b1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:31.254109 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:31.254806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:31.255085 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:31.255359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] PATH_INFO: `/v3/users/254265aafb4c455ea472a1a4c4507ec2` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:31.256639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=254265aafb4c455ea472a1a4c4507ec2)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:31.258903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:31.482677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:29:31.592281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '254265aafb4c455ea472a1a4c4507ec2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:31.593156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '254265aafb4c455ea472a1a4c4507ec2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:31.593978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '254265aafb4c455ea472a1a4c4507ec2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:31.607772 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '254265aafb4c455ea472a1a4c4507ec2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:31.637255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '254265aafb4c455ea472a1a4c4507ec2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:31.732814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1080/2194] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:29:31 2022] DELETE /identity/v3/users/254265aafb4c455ea472a1a4c4507ec2 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:29:31.738745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a8c2a-ec2b-46aa-b553-cd71dc354963 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:31.739317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a8c2a-ec2b-46aa-b553-cd71dc354963 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:31.739539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a8c2a-ec2b-46aa-b553-cd71dc354963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:31.763807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-756a8c2a-ec2b-46aa-b553-cd71dc354963 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:29:31.764977 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-756a8c2a-ec2b-46aa-b553-cd71dc354963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:31.881239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1115/2195] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:29:31.893794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:31.895679 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:31.980094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1c0b1-c26c-420e-b1e1-6fa300d5a487 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:31.984488 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65df3c7-ebe7-4874-8cc7-71264dbb2ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:31.985123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65df3c7-ebe7-4874-8cc7-71264dbb2ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:31.985401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65df3c7-ebe7-4874-8cc7-71264dbb2ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:31.985672 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65df3c7-ebe7-4874-8cc7-71264dbb2ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:31.997289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65df3c7-ebe7-4874-8cc7-71264dbb2ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:31.998819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65df3c7-ebe7-4874-8cc7-71264dbb2ab0 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:32.011188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1081/2196] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:29:32.714281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a8c2a-ec2b-46aa-b553-cd71dc354963 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:29:32.716631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-756a8c2a-ec2b-46aa-b553-cd71dc354963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:32.753094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a8c2a-ec2b-46aa-b553-cd71dc354963 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:29:32.765814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:29:32.765814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:32.765814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:32.765814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] PATH_INFO: `/v3/projects/e75440603c194725989299f0928d1c40` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:32.766938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e75440603c194725989299f0928d1c40)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:29:32.772168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:29:32.980877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e75440603c194725989299f0928d1c40', 'request_id': 'req-eb2385df-1e69-4d6e-9426-67182bd8a375'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:32.987557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:29:33.002416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e75440603c194725989299f0928d1c40', 'request_id': 'req-eb2385df-1e69-4d6e-9426-67182bd8a375'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:33.015855 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e75440603c194725989299f0928d1c40', 'request_id': 'req-eb2385df-1e69-4d6e-9426-67182bd8a375'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:33.152878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] The token cache is being invalidate because project e75440603c194725989299f0928d1c40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:29:33.153299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:33.154302 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:29:33.156382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1116/2197] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:29:32 2022] DELETE /identity/v3/projects/e75440603c194725989299f0928d1c40 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:29:34.324683 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc04475-dddc-43a9-a426-00c5be281b15 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:29:34.325382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc04475-dddc-43a9-a426-00c5be281b15 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:29:34.325382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc04475-dddc-43a9-a426-00c5be281b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:29:34.347071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1bc04475-dddc-43a9-a426-00c5be281b15 None None] MFA Rules not processed for user `f46cbc7de18c4f9fb6f54e96be53fea0`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:29:34.348517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1bc04475-dddc-43a9-a426-00c5be281b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:29:34.414145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1082/2198] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:29:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:29:43.544999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=6,keystone:DELETE=3,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:29:44.406534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=6,keystone:SELECT=70,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:30:31.587794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:31.587794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:31.612247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc04475-dddc-43a9-a426-00c5be281b15 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:31.614171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1bc04475-dddc-43a9-a426-00c5be281b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:31.694666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2385df-1e69-4d6e-9426-67182bd8a375 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:31.701464 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:31.702745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:31.703212 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:31.703669 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] PATH_INFO: `/v3/users/9478834ef55643d79abf01a7cd23ed71` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:31.707430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9478834ef55643d79abf01a7cd23ed71)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:31.711256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:31.711932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc04475-dddc-43a9-a426-00c5be281b15 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:31.716841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:31.717520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:31.717894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:31.718159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] PATH_INFO: `/v3/users/371d90bbb0e0461aafc024284acf7590` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:31.719547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] RBAC: Authorizing `identity:delete_user(user_id=371d90bbb0e0461aafc024284acf7590)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:31.722249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:32.043993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9478834ef55643d79abf01a7cd23ed71'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:32.045003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9478834ef55643d79abf01a7cd23ed71'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:32.045920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9478834ef55643d79abf01a7cd23ed71'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:32.058137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371d90bbb0e0461aafc024284acf7590'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:32.063589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371d90bbb0e0461aafc024284acf7590'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:32.063589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371d90bbb0e0461aafc024284acf7590'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:32.069143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371d90bbb0e0461aafc024284acf7590'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:32.083574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-649beab8-a691-4828-82fa-d95c4abf492e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371d90bbb0e0461aafc024284acf7590'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:32.167674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9478834ef55643d79abf01a7cd23ed71'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:32.178421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-2d67aec6-dbec-48aa-bd66-825230774551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9478834ef55643d79abf01a7cd23ed71'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:32.201928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1117/2199] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:30:31 2022] DELETE /identity/v3/users/9478834ef55643d79abf01a7cd23ed71 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:30:32.208338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540db55-a992-4f0a-839f-06bdcf0f46fe None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:32.211955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540db55-a992-4f0a-839f-06bdcf0f46fe None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:32.211955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540db55-a992-4f0a-839f-06bdcf0f46fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:32.237708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1083/2200] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:30:31 2022] DELETE /identity/v3/users/371d90bbb0e0461aafc024284acf7590 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:30:32.244324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a8c81-68f3-4e14-adf8-b826190a6041 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:32.244789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a8c81-68f3-4e14-adf8-b826190a6041 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:32.245200 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a8c81-68f3-4e14-adf8-b826190a6041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:32.254212 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-1540db55-a992-4f0a-839f-06bdcf0f46fe None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:30:32.256246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1540db55-a992-4f0a-839f-06bdcf0f46fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:32.268704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-cf4a8c81-68f3-4e14-adf8-b826190a6041 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:30:32.270026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cf4a8c81-68f3-4e14-adf8-b826190a6041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:32.348168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1084/2201] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:30:32.366872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1118/2202] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:30:32.367182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a8c81-68f3-4e14-adf8-b826190a6041 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:32.367182 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cf4a8c81-68f3-4e14-adf8-b826190a6041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:32.375580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540db55-a992-4f0a-839f-06bdcf0f46fe None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:32.377898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1540db55-a992-4f0a-839f-06bdcf0f46fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:32.400259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:30:32.407976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:30:32.449366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a8c81-68f3-4e14-adf8-b826190a6041 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:32.455688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b1356-1d41-48ba-9564-0046f021d218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:32.456325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b1356-1d41-48ba-9564-0046f021d218 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:32.456601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b1356-1d41-48ba-9564-0046f021d218 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:32.456869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b1356-1d41-48ba-9564-0046f021d218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:32.465363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540db55-a992-4f0a-839f-06bdcf0f46fe None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:32.467754 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b1356-1d41-48ba-9564-0046f021d218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:32.468911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b1356-1d41-48ba-9564-0046f021d218 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:32.472678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbf588-3ed5-4c4e-85b4-9611f3f9c603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:32.473720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbf588-3ed5-4c4e-85b4-9611f3f9c603 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:32.474222 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbf588-3ed5-4c4e-85b4-9611f3f9c603 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:32.474730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbf588-3ed5-4c4e-85b4-9611f3f9c603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:32.484131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1085/2203] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:30:32.489739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbf588-3ed5-4c4e-85b4-9611f3f9c603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:32.494601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbf588-3ed5-4c4e-85b4-9611f3f9c603 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:32.503853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1119/2204] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:30:33.354949 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b1356-1d41-48ba-9564-0046f021d218 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:33.357554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-220b1356-1d41-48ba-9564-0046f021d218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:33.362533 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbf588-3ed5-4c4e-85b4-9611f3f9c603 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:33.365208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4abbf588-3ed5-4c4e-85b4-9611f3f9c603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:33.384459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b1356-1d41-48ba-9564-0046f021d218 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:33.397681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbf588-3ed5-4c4e-85b4-9611f3f9c603 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:33.405250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:33.405858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:33.406132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:33.406430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] PATH_INFO: `/v3/projects/e9ecdd9e74e0435c8bd03065b5225403` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:33.406850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:33.407998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:33.408207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ecdd9e74e0435c8bd03065b5225403)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:33.408525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:33.408974 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] PATH_INFO: `/v3/projects/279398a188ca4c7b8c8b502664aac429` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:33.410072 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:33.411277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=279398a188ca4c7b8c8b502664aac429)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:33.415340 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:33.596357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ecdd9e74e0435c8bd03065b5225403', 'request_id': 'req-c57405ad-0a69-487e-af2e-4ad82489724d'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:33.609468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ecdd9e74e0435c8bd03065b5225403', 'request_id': 'req-c57405ad-0a69-487e-af2e-4ad82489724d'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:33.625162 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '279398a188ca4c7b8c8b502664aac429', 'request_id': 'req-64df3f31-f512-4d2b-9a11-b62253d3db22'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:33.633288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '279398a188ca4c7b8c8b502664aac429', 'request_id': 'req-64df3f31-f512-4d2b-9a11-b62253d3db22'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:33.640538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '279398a188ca4c7b8c8b502664aac429', 'request_id': 'req-64df3f31-f512-4d2b-9a11-b62253d3db22'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:33.722077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ecdd9e74e0435c8bd03065b5225403', 'request_id': 'req-c57405ad-0a69-487e-af2e-4ad82489724d'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:33.735565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] The token cache is being invalidate because project e9ecdd9e74e0435c8bd03065b5225403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:30:33.736006 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:33.737007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:33.738835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1086/2205] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:30:33 2022] DELETE /identity/v3/projects/e9ecdd9e74e0435c8bd03065b5225403 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:30:33.743529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:33.745176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:33.816202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] The token cache is being invalidate because project 279398a188ca4c7b8c8b502664aac429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:30:33.816611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:33.817379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-64df3f31-f512-4d2b-9a11-b62253d3db22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:33.819130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1120/2206] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:30:33 2022] DELETE /identity/v3/projects/279398a188ca4c7b8c8b502664aac429 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:30:33.827233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498605-d43d-4ef2-8b13-74ff482f0c28 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:33.827587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498605-d43d-4ef2-8b13-74ff482f0c28 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:33.827913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498605-d43d-4ef2-8b13-74ff482f0c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:33.833300 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57405ad-0a69-487e-af2e-4ad82489724d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:33.861902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d2498605-d43d-4ef2-8b13-74ff482f0c28 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:30:33.863736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d2498605-d43d-4ef2-8b13-74ff482f0c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:33.910170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:30:33.924627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:33.925467 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:33.925855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:33.926272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] PATH_INFO: `/v3/users/7192defda870422b9e5875bb63bbbe4f` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:33.927884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7192defda870422b9e5875bb63bbbe4f)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:33.930731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:33.960014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1121/2207] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:30:33.970063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c298ad-7d46-4b3c-b684-1ecf86d3cf38 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:33.970544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c298ad-7d46-4b3c-b684-1ecf86d3cf38 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:33.970971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c298ad-7d46-4b3c-b684-1ecf86d3cf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:33.995123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-38c298ad-7d46-4b3c-b684-1ecf86d3cf38 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:30:33.996517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-38c298ad-7d46-4b3c-b684-1ecf86d3cf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:34.006225 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1122/2208] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:30:34.011269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c298ad-7d46-4b3c-b684-1ecf86d3cf38 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:34.021996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c298ad-7d46-4b3c-b684-1ecf86d3cf38 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:34.027277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d01d19-4e81-46b6-9992-5a1148cd6c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:34.028108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d01d19-4e81-46b6-9992-5a1148cd6c2a admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:34.028530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d01d19-4e81-46b6-9992-5a1148cd6c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:34.028853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d01d19-4e81-46b6-9992-5a1148cd6c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:34.029672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d01d19-4e81-46b6-9992-5a1148cd6c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:30:34.030020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d01d19-4e81-46b6-9992-5a1148cd6c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:34.032094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d01d19-4e81-46b6-9992-5a1148cd6c2a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:34.046702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1123/2209] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:30:34.053044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d01d19-4e81-46b6-9992-5a1148cd6c2a admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:34.064802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d01d19-4e81-46b6-9992-5a1148cd6c2a admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:34.070339 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f83857-e377-47ac-bda9-9ea3ebf84da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:34.071015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f83857-e377-47ac-bda9-9ea3ebf84da8 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:34.071297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f83857-e377-47ac-bda9-9ea3ebf84da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:34.071572 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f83857-e377-47ac-bda9-9ea3ebf84da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:34.072245 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f83857-e377-47ac-bda9-9ea3ebf84da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:34.074165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f83857-e377-47ac-bda9-9ea3ebf84da8 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:34.206760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7192defda870422b9e5875bb63bbbe4f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:34.208453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7192defda870422b9e5875bb63bbbe4f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:34.209402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7192defda870422b9e5875bb63bbbe4f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:34.222803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7192defda870422b9e5875bb63bbbe4f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:34.224218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1124/2210] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:30:34 2022] POST /identity/v3/projects => generated 365 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:30:34.231044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f83857-e377-47ac-bda9-9ea3ebf84da8 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:34.233101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c2f83857-e377-47ac-bda9-9ea3ebf84da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:34.242329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:30:34.249546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7192defda870422b9e5875bb63bbbe4f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:34.334942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f83857-e377-47ac-bda9-9ea3ebf84da8 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:34.341926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a94143-8fb7-4ad8-b36d-11e7056d9806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:34.342783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a94143-8fb7-4ad8-b36d-11e7056d9806 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:34.343045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a94143-8fb7-4ad8-b36d-11e7056d9806 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:34.350907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a94143-8fb7-4ad8-b36d-11e7056d9806 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:34.350907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a94143-8fb7-4ad8-b36d-11e7056d9806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:34.350907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a94143-8fb7-4ad8-b36d-11e7056d9806 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:34.351238 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1087/2211] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:30:33 2022] DELETE /identity/v3/users/7192defda870422b9e5875bb63bbbe4f => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:30:34.442646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1125/2212] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:30:34 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:30:34.446030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:34.549259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5f01f-9970-47c2-bddb-db4e70b76ab5 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:34.554399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14276638-449b-4163-98b3-d304bc91cec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:34.555077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14276638-449b-4163-98b3-d304bc91cec8 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:34.555339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14276638-449b-4163-98b3-d304bc91cec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:34.555632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14276638-449b-4163-98b3-d304bc91cec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:34.557752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14276638-449b-4163-98b3-d304bc91cec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:34.560143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14276638-449b-4163-98b3-d304bc91cec8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:34.576936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1088/2213] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:30:34.580482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a94143-8fb7-4ad8-b36d-11e7056d9806 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:34.603024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a94143-8fb7-4ad8-b36d-11e7056d9806 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:34.615796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9470c-b0ec-4c70-8bc3-574df8625267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:34.616379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9470c-b0ec-4c70-8bc3-574df8625267 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:34.616823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9470c-b0ec-4c70-8bc3-574df8625267 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:34.617025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9470c-b0ec-4c70-8bc3-574df8625267 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:34.617993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9470c-b0ec-4c70-8bc3-574df8625267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:34.621109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9470c-b0ec-4c70-8bc3-574df8625267 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:34.639994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14276638-449b-4163-98b3-d304bc91cec8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:34.642116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1126/2214] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:30:34.653634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14276638-449b-4163-98b3-d304bc91cec8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:34.663810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bff5e-1040-486d-8c70-b95db7425d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:34.664924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bff5e-1040-486d-8c70-b95db7425d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:34.665400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bff5e-1040-486d-8c70-b95db7425d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:34.667183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bff5e-1040-486d-8c70-b95db7425d4c admin admin] PATH_INFO: `/v3/projects/658ec261e291457ca3bbf1f62c9ba551/users/8e045711ea2742c498e0bcd4017b52cd/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:34.694763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bff5e-1040-486d-8c70-b95db7425d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=658ec261e291457ca3bbf1f62c9ba551, user_id=8e045711ea2742c498e0bcd4017b52cd, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:34.698289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bff5e-1040-486d-8c70-b95db7425d4c admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:34.789278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1089/2215] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:30:34 2022] PUT /identity/v3/projects/658ec261e291457ca3bbf1f62c9ba551/users/8e045711ea2742c498e0bcd4017b52cd/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:30:34.805367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a510272-2f81-4dfc-996c-0d55d6ecd66a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:34.805745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a510272-2f81-4dfc-996c-0d55d6ecd66a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:34.806094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a510272-2f81-4dfc-996c-0d55d6ecd66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:34.842898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2a510272-2f81-4dfc-996c-0d55d6ecd66a None None] MFA Rules not processed for user `8e045711ea2742c498e0bcd4017b52cd`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:30:34.844161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2a510272-2f81-4dfc-996c-0d55d6ecd66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:34.886908 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bff5e-1040-486d-8c70-b95db7425d4c admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:34.890097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-203bff5e-1040-486d-8c70-b95db7425d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:34.974725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1127/2216] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:30:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:30:34.985888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a510272-2f81-4dfc-996c-0d55d6ecd66a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:34.987639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:30:34.988188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2a510272-2f81-4dfc-996c-0d55d6ecd66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:35.019175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bff5e-1040-486d-8c70-b95db7425d4c admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:35.027991 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:35.028840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:35.029201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:35.029536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] PATH_INFO: `/v3/projects/6ab980c9c48349a7ab98d58ec3c57196` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:35.030927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ab980c9c48349a7ab98d58ec3c57196)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:35.034313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:35.068858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a510272-2f81-4dfc-996c-0d55d6ecd66a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:35.076720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee00d6-3c7d-4090-b637-4197e63c8d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:35.077902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee00d6-3c7d-4090-b637-4197e63c8d96 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:35.078515 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee00d6-3c7d-4090-b637-4197e63c8d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:35.079105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee00d6-3c7d-4090-b637-4197e63c8d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:35.090205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee00d6-3c7d-4090-b637-4197e63c8d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:35.091782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee00d6-3c7d-4090-b637-4197e63c8d96 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:35.103996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1128/2217] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:30:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:30:35.191849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ab980c9c48349a7ab98d58ec3c57196', 'request_id': 'req-44c68ad8-2875-4cdd-8a3c-4f79da508208'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:35.202202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ab980c9c48349a7ab98d58ec3c57196', 'request_id': 'req-44c68ad8-2875-4cdd-8a3c-4f79da508208'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:35.310293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ab980c9c48349a7ab98d58ec3c57196', 'request_id': 'req-44c68ad8-2875-4cdd-8a3c-4f79da508208'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:35.339145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] The token cache is being invalidate because project 6ab980c9c48349a7ab98d58ec3c57196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:30:35.339145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:35.339145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-44c68ad8-2875-4cdd-8a3c-4f79da508208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:30:35.339910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1090/2218] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:30:34 2022] DELETE /identity/v3/projects/6ab980c9c48349a7ab98d58ec3c57196 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:30:35.348096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88835a23-e0bc-4dea-af22-845bf3a1586c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:35.349740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88835a23-e0bc-4dea-af22-845bf3a1586c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:35.350755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88835a23-e0bc-4dea-af22-845bf3a1586c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:35.389242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-88835a23-e0bc-4dea-af22-845bf3a1586c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:30:35.390862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-88835a23-e0bc-4dea-af22-845bf3a1586c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:35.408174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:30:35.498101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1129/2219] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:30:35.508848 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9360673-5025-4911-b289-63f8f10dad4d None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:35.510538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9360673-5025-4911-b289-63f8f10dad4d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:35.510538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9360673-5025-4911-b289-63f8f10dad4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:35.538100 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-e9360673-5025-4911-b289-63f8f10dad4d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:30:35.540212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e9360673-5025-4911-b289-63f8f10dad4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:35.555178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1091/2220] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:30:35.560990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88835a23-e0bc-4dea-af22-845bf3a1586c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:35.577005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88835a23-e0bc-4dea-af22-845bf3a1586c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:35.585472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dff5eb-3f62-45e7-9a15-e3b7252b64cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:35.586175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dff5eb-3f62-45e7-9a15-e3b7252b64cd admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:35.586520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dff5eb-3f62-45e7-9a15-e3b7252b64cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:35.586883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dff5eb-3f62-45e7-9a15-e3b7252b64cd admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:35.588003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dff5eb-3f62-45e7-9a15-e3b7252b64cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:30:35.588514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dff5eb-3f62-45e7-9a15-e3b7252b64cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:35.590950 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dff5eb-3f62-45e7-9a15-e3b7252b64cd admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:35.608445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1130/2221] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:30:35.613631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9360673-5025-4911-b289-63f8f10dad4d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:35.630500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9360673-5025-4911-b289-63f8f10dad4d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:35.638281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24a2aa-e50f-4cec-b9b9-87d8770d544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:35.639199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24a2aa-e50f-4cec-b9b9-87d8770d544e admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:35.639604 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24a2aa-e50f-4cec-b9b9-87d8770d544e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:35.640079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24a2aa-e50f-4cec-b9b9-87d8770d544e admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:35.642254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24a2aa-e50f-4cec-b9b9-87d8770d544e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:35.645985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24a2aa-e50f-4cec-b9b9-87d8770d544e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:35.778554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1092/2222] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:30:35 2022] POST /identity/v3/projects => generated 357 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:30:35.784265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dff5eb-3f62-45e7-9a15-e3b7252b64cd admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:35.797810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dff5eb-3f62-45e7-9a15-e3b7252b64cd admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:35.897140 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665a019-b5ba-41eb-8ea9-88789342003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:35.898854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665a019-b5ba-41eb-8ea9-88789342003d admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:35.899290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665a019-b5ba-41eb-8ea9-88789342003d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:35.899808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665a019-b5ba-41eb-8ea9-88789342003d admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:35.900861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665a019-b5ba-41eb-8ea9-88789342003d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:35.905572 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665a019-b5ba-41eb-8ea9-88789342003d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:36.026011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1131/2223] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:30:35 2022] POST /identity/v3/users => generated 279 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:30:36.032508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24a2aa-e50f-4cec-b9b9-87d8770d544e admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:36.051908 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24a2aa-e50f-4cec-b9b9-87d8770d544e admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:36.061355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af83b6-6339-4905-a828-70f015ddd1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:36.062511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af83b6-6339-4905-a828-70f015ddd1fd admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:36.062990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af83b6-6339-4905-a828-70f015ddd1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:36.064773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af83b6-6339-4905-a828-70f015ddd1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:36.066044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af83b6-6339-4905-a828-70f015ddd1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:36.068269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af83b6-6339-4905-a828-70f015ddd1fd admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:36.088605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1093/2224] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:30:36.093774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665a019-b5ba-41eb-8ea9-88789342003d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:36.112500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665a019-b5ba-41eb-8ea9-88789342003d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:36.118281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765de9f-b7df-4206-bda7-eb52e722bfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:36.120590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765de9f-b7df-4206-bda7-eb52e722bfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:36.120590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765de9f-b7df-4206-bda7-eb52e722bfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:36.120590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765de9f-b7df-4206-bda7-eb52e722bfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:36.120590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765de9f-b7df-4206-bda7-eb52e722bfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:36.122745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765de9f-b7df-4206-bda7-eb52e722bfd0 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:36.140101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1132/2225] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:30:36.147456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af83b6-6339-4905-a828-70f015ddd1fd admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:36.170091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af83b6-6339-4905-a828-70f015ddd1fd admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:36.184092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d0ddd-2e97-453a-98c2-d5f285fb156d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:36.185165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d0ddd-2e97-453a-98c2-d5f285fb156d admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:36.185640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d0ddd-2e97-453a-98c2-d5f285fb156d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:36.186125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d0ddd-2e97-453a-98c2-d5f285fb156d admin admin] PATH_INFO: `/v3/projects/add76b4d62654be391a54dc8219f8433/users/b329ab1485634831882c601527a4b643/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:36.212582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d0ddd-2e97-453a-98c2-d5f285fb156d admin admin] RBAC: Authorizing `identity:create_grant(project_id=add76b4d62654be391a54dc8219f8433, user_id=b329ab1485634831882c601527a4b643, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:36.216024 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d0ddd-2e97-453a-98c2-d5f285fb156d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:36.336910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1094/2226] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:30:36 2022] PUT /identity/v3/projects/add76b4d62654be391a54dc8219f8433/users/b329ab1485634831882c601527a4b643/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:30:36.357020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2984ed5-551e-4397-a136-87b2b0d49842 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:36.357390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2984ed5-551e-4397-a136-87b2b0d49842 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:36.357635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2984ed5-551e-4397-a136-87b2b0d49842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:36.410413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f2984ed5-551e-4397-a136-87b2b0d49842 None None] MFA Rules not processed for user `b329ab1485634831882c601527a4b643`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:30:36.412852 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f2984ed5-551e-4397-a136-87b2b0d49842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:36.512630 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1133/2227] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:30:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:30:36.521420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d0ddd-2e97-453a-98c2-d5f285fb156d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:36.620433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d0ddd-2e97-453a-98c2-d5f285fb156d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:36.627599 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4c714-9bf6-435d-b7eb-8a814d81d3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:36.629505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4c714-9bf6-435d-b7eb-8a814d81d3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:36.629505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4c714-9bf6-435d-b7eb-8a814d81d3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:36.629505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4c714-9bf6-435d-b7eb-8a814d81d3a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:36.629924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4c714-9bf6-435d-b7eb-8a814d81d3a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:36.631747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4c714-9bf6-435d-b7eb-8a814d81d3a9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:36.652329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:30:36.743992 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1095/2228] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:30:36 2022] POST /identity/v3/projects => generated 357 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:30:36.749214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2984ed5-551e-4397-a136-87b2b0d49842 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:36.768700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2984ed5-551e-4397-a136-87b2b0d49842 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:36.781878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:30:36.858405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec30426-c437-4640-a40a-bb16547b3b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:36.859216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec30426-c437-4640-a40a-bb16547b3b3e admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:36.859625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec30426-c437-4640-a40a-bb16547b3b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:36.860657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec30426-c437-4640-a40a-bb16547b3b3e admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:36.861347 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec30426-c437-4640-a40a-bb16547b3b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:36.863714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec30426-c437-4640-a40a-bb16547b3b3e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:36.982662 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1134/2229] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:30:36 2022] POST /identity/v3/users => generated 285 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:30:36.988450 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4c714-9bf6-435d-b7eb-8a814d81d3a9 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:37.004412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4c714-9bf6-435d-b7eb-8a814d81d3a9 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:37.011232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e48509-24b4-434d-8c9e-9079e15b62c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:37.017031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e48509-24b4-434d-8c9e-9079e15b62c9 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:37.017031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e48509-24b4-434d-8c9e-9079e15b62c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:37.017031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e48509-24b4-434d-8c9e-9079e15b62c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:37.017031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e48509-24b4-434d-8c9e-9079e15b62c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:37.017031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e48509-24b4-434d-8c9e-9079e15b62c9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:37.032796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1096/2230] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:30:37.037734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec30426-c437-4640-a40a-bb16547b3b3e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:37.053279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec30426-c437-4640-a40a-bb16547b3b3e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:37.059145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69d020-48d1-42c8-825b-1b90b6dd992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:37.059838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69d020-48d1-42c8-825b-1b90b6dd992c admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:37.060120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69d020-48d1-42c8-825b-1b90b6dd992c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:37.060397 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69d020-48d1-42c8-825b-1b90b6dd992c admin admin] PATH_INFO: `/v3/projects/14ed0613484e4bcd971267bfd6c4b8d2/users/759dcbd3433246afb6d5a19ea365b045/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:37.080933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69d020-48d1-42c8-825b-1b90b6dd992c admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ed0613484e4bcd971267bfd6c4b8d2, user_id=759dcbd3433246afb6d5a19ea365b045, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:37.085952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69d020-48d1-42c8-825b-1b90b6dd992c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:37.211219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1135/2231] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:30:37 2022] PUT /identity/v3/projects/14ed0613484e4bcd971267bfd6c4b8d2/users/759dcbd3433246afb6d5a19ea365b045/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:30:37.215946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e48509-24b4-434d-8c9e-9079e15b62c9 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:37.231738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e48509-24b4-434d-8c9e-9079e15b62c9 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:37.320185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf6af3-a05e-4ab1-b844-1e452b8960c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:37.321283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf6af3-a05e-4ab1-b844-1e452b8960c5 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:37.321799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf6af3-a05e-4ab1-b844-1e452b8960c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:37.322275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf6af3-a05e-4ab1-b844-1e452b8960c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:37.323596 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf6af3-a05e-4ab1-b844-1e452b8960c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:37.326792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf6af3-a05e-4ab1-b844-1e452b8960c5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:37.352922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1097/2232] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:30:37.354261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69d020-48d1-42c8-825b-1b90b6dd992c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:37.367452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69d020-48d1-42c8-825b-1b90b6dd992c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:37.373982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be6c39-cb51-49a4-972e-173037de77f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:37.374716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be6c39-cb51-49a4-972e-173037de77f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:37.374991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be6c39-cb51-49a4-972e-173037de77f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:37.375263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be6c39-cb51-49a4-972e-173037de77f4 admin admin] PATH_INFO: `/v3/projects/14ed0613484e4bcd971267bfd6c4b8d2/users/759dcbd3433246afb6d5a19ea365b045/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:37.377519 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be6c39-cb51-49a4-972e-173037de77f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ed0613484e4bcd971267bfd6c4b8d2, user_id=759dcbd3433246afb6d5a19ea365b045, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:37.381719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be6c39-cb51-49a4-972e-173037de77f4 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:37.394583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1136/2233] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:30:37 2022] PUT /identity/v3/projects/14ed0613484e4bcd971267bfd6c4b8d2/users/759dcbd3433246afb6d5a19ea365b045/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:30:37.405484 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afa9ee-e88c-47b5-ba96-bdebaf29e902 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:37.405918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afa9ee-e88c-47b5-ba96-bdebaf29e902 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:37.406307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afa9ee-e88c-47b5-ba96-bdebaf29e902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:37.451594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-31afa9ee-e88c-47b5-ba96-bdebaf29e902 None None] MFA Rules not processed for user `759dcbd3433246afb6d5a19ea365b045`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:30:37.453170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-31afa9ee-e88c-47b5-ba96-bdebaf29e902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:37.579508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1098/2234] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:30:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:30:37.589805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be6c39-cb51-49a4-972e-173037de77f4 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:37.592540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-62be6c39-cb51-49a4-972e-173037de77f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:30:37.684936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be6c39-cb51-49a4-972e-173037de77f4 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:37.692062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b3b1e-3f91-47ac-8b3e-06cfd4692c49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:37.692914 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b3b1e-3f91-47ac-8b3e-06cfd4692c49 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:37.693360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b3b1e-3f91-47ac-8b3e-06cfd4692c49 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:37.693963 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b3b1e-3f91-47ac-8b3e-06cfd4692c49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:37.706799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b3b1e-3f91-47ac-8b3e-06cfd4692c49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:37.709229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b3b1e-3f91-47ac-8b3e-06cfd4692c49 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:37.724867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1137/2235] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:30:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:30:37.988738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afa9ee-e88c-47b5-ba96-bdebaf29e902 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:30:38.001967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afa9ee-e88c-47b5-ba96-bdebaf29e902 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:30:38.010666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a3e0b-e2c2-4cba-82ad-dd3eec070715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:30:38.010666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a3e0b-e2c2-4cba-82ad-dd3eec070715 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:30:38.010666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a3e0b-e2c2-4cba-82ad-dd3eec070715 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:30:38.010666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a3e0b-e2c2-4cba-82ad-dd3eec070715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:30:38.093813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a3e0b-e2c2-4cba-82ad-dd3eec070715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:30:38.097363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a3e0b-e2c2-4cba-82ad-dd3eec070715 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:30:38.116301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1099/2236] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:30:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:30:47.717908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=65,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:30:48.107490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=98 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:04.564872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b3b1e-3f91-47ac-8b3e-06cfd4692c49 service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:04.647925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b3b1e-3f91-47ac-8b3e-06cfd4692c49 service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:04.653473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:04.654133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:04.654413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:04.654725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] PATH_INFO: `/v3/users/b329ab1485634831882c601527a4b643` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:04.656595 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b329ab1485634831882c601527a4b643)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:04.658969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:04.894764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b329ab1485634831882c601527a4b643'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:04.895387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b329ab1485634831882c601527a4b643'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:04.896160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b329ab1485634831882c601527a4b643'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:04.989737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b329ab1485634831882c601527a4b643'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:05.000892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b329ab1485634831882c601527a4b643'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:05.023200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1138/2237] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:31:04 2022] DELETE /identity/v3/users/b329ab1485634831882c601527a4b643 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:05.028978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdabd08-5ef0-461a-a026-7d7928bd3845 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:05.029716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdabd08-5ef0-461a-a026-7d7928bd3845 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:05.029716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdabd08-5ef0-461a-a026-7d7928bd3845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:05.056633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-fcdabd08-5ef0-461a-a026-7d7928bd3845 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:05.056633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fcdabd08-5ef0-461a-a026-7d7928bd3845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:05.133428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1100/2238] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:31:05.145804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:05.148059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:05.227962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804e264-b908-4358-9189-1a916f36791f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:05.231757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a3fa8-983e-41c0-823f-d8b4a80ff166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:05.232393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a3fa8-983e-41c0-823f-d8b4a80ff166 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:05.232705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a3fa8-983e-41c0-823f-d8b4a80ff166 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:05.232984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a3fa8-983e-41c0-823f-d8b4a80ff166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:05.245806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a3fa8-983e-41c0-823f-d8b4a80ff166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:05.247441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a3fa8-983e-41c0-823f-d8b4a80ff166 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:05.259700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1139/2239] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:05.852993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdabd08-5ef0-461a-a026-7d7928bd3845 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:05.857794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fcdabd08-5ef0-461a-a026-7d7928bd3845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:05.873923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdabd08-5ef0-461a-a026-7d7928bd3845 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:05.880293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:05.880952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:05.881235 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:05.881506 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] PATH_INFO: `/v3/projects/add76b4d62654be391a54dc8219f8433` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:05.882757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=add76b4d62654be391a54dc8219f8433)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:05.885025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:06.011994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add76b4d62654be391a54dc8219f8433', 'request_id': 'req-9786f17c-eab8-4dd6-aaa7-455b8edd0046'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:06.020046 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add76b4d62654be391a54dc8219f8433', 'request_id': 'req-9786f17c-eab8-4dd6-aaa7-455b8edd0046'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:06.110327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add76b4d62654be391a54dc8219f8433', 'request_id': 'req-9786f17c-eab8-4dd6-aaa7-455b8edd0046'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:06.125364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] The token cache is being invalidate because project add76b4d62654be391a54dc8219f8433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:31:06.125894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:06.127123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:06.129832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1101/2240] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:31:05 2022] DELETE /identity/v3/projects/add76b4d62654be391a54dc8219f8433 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:31:06.135664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a3fa8-983e-41c0-823f-d8b4a80ff166 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:06.148681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a3fa8-983e-41c0-823f-d8b4a80ff166 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:06.179414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:06.232007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:06.232656 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:06.232939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:06.233502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] PATH_INFO: `/v3/users/759dcbd3433246afb6d5a19ea365b045` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:06.234553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=759dcbd3433246afb6d5a19ea365b045)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:06.236667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:06.653532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '759dcbd3433246afb6d5a19ea365b045'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:06.654655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '759dcbd3433246afb6d5a19ea365b045'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:06.655639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '759dcbd3433246afb6d5a19ea365b045'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:06.755831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '759dcbd3433246afb6d5a19ea365b045'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:06.765120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-714a64e4-7ca0-493f-a780-d27e9e1b5c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '759dcbd3433246afb6d5a19ea365b045'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:06.787846 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1140/2241] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:31:06 2022] DELETE /identity/v3/users/759dcbd3433246afb6d5a19ea365b045 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:07.290635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:07.294520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:07.320725 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786f17c-eab8-4dd6-aaa7-455b8edd0046 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:07.413181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:07.413845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:07.414120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:07.414527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] PATH_INFO: `/v3/projects/14ed0613484e4bcd971267bfd6c4b8d2` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:07.415810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14ed0613484e4bcd971267bfd6c4b8d2)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:07.417782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:07.523608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:07.545260 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14ed0613484e4bcd971267bfd6c4b8d2', 'request_id': 'req-187923b0-acbc-46ba-8c85-3ae0d7ac6092'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:07.553319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14ed0613484e4bcd971267bfd6c4b8d2', 'request_id': 'req-187923b0-acbc-46ba-8c85-3ae0d7ac6092'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:07.635546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14ed0613484e4bcd971267bfd6c4b8d2', 'request_id': 'req-187923b0-acbc-46ba-8c85-3ae0d7ac6092'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:07.651757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] The token cache is being invalidate because project 14ed0613484e4bcd971267bfd6c4b8d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:31:07.652218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:07.653015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-187923b0-acbc-46ba-8c85-3ae0d7ac6092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:07.654834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1102/2242] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:31:07 2022] DELETE /identity/v3/projects/14ed0613484e4bcd971267bfd6c4b8d2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:07.661320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56d9c0-de24-4d1d-ab95-72542296421a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:07.661993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56d9c0-de24-4d1d-ab95-72542296421a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:07.661993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56d9c0-de24-4d1d-ab95-72542296421a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:07.686073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6e56d9c0-de24-4d1d-ab95-72542296421a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:07.687463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6e56d9c0-de24-4d1d-ab95-72542296421a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:07.773227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1141/2243] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:31:07.784555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fdc5b-83d4-4d6f-b6e7-674cc57f5bf8 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:07.784936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fdc5b-83d4-4d6f-b6e7-674cc57f5bf8 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:07.785212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fdc5b-83d4-4d6f-b6e7-674cc57f5bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:07.808386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-f68fdc5b-83d4-4d6f-b6e7-674cc57f5bf8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:07.809649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f68fdc5b-83d4-4d6f-b6e7-674cc57f5bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:07.819300 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1103/2244] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:31:07.824302 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56d9c0-de24-4d1d-ab95-72542296421a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:07.835875 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56d9c0-de24-4d1d-ab95-72542296421a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:07.841074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbd801-6b48-44aa-beda-0838ee9a7d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:07.841719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbd801-6b48-44aa-beda-0838ee9a7d50 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:07.841998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbd801-6b48-44aa-beda-0838ee9a7d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:07.842270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbd801-6b48-44aa-beda-0838ee9a7d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:07.843006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbd801-6b48-44aa-beda-0838ee9a7d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:31:07.843311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbd801-6b48-44aa-beda-0838ee9a7d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:07.845645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbd801-6b48-44aa-beda-0838ee9a7d50 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:07.860042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1142/2245] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:31:07.865392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fdc5b-83d4-4d6f-b6e7-674cc57f5bf8 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:07.883710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fdc5b-83d4-4d6f-b6e7-674cc57f5bf8 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:07.892884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69b8af-030f-4acd-88cd-872e0bdcd16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:07.893909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69b8af-030f-4acd-88cd-872e0bdcd16d admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:07.894396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69b8af-030f-4acd-88cd-872e0bdcd16d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:07.894885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69b8af-030f-4acd-88cd-872e0bdcd16d admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:07.895959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69b8af-030f-4acd-88cd-872e0bdcd16d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:07.899691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69b8af-030f-4acd-88cd-872e0bdcd16d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:08.009683 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1104/2246] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:31:07 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:31:08.013941 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbd801-6b48-44aa-beda-0838ee9a7d50 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:08.026221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbd801-6b48-44aa-beda-0838ee9a7d50 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:08.070606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:08.121814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a6f8f-f7d2-4665-bfb1-a239872c12d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:08.123003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a6f8f-f7d2-4665-bfb1-a239872c12d0 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:08.123599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a6f8f-f7d2-4665-bfb1-a239872c12d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:08.124041 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a6f8f-f7d2-4665-bfb1-a239872c12d0 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:08.126248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a6f8f-f7d2-4665-bfb1-a239872c12d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:08.129777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a6f8f-f7d2-4665-bfb1-a239872c12d0 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:08.251979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1143/2247] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:31:08 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:31:08.256831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69b8af-030f-4acd-88cd-872e0bdcd16d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:08.272380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69b8af-030f-4acd-88cd-872e0bdcd16d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:08.278051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab9d5-c623-4e66-87f3-444ddedb5e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:08.278777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab9d5-c623-4e66-87f3-444ddedb5e2f admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:08.279060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab9d5-c623-4e66-87f3-444ddedb5e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:08.279492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab9d5-c623-4e66-87f3-444ddedb5e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:08.280521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dab9d5-c623-4e66-87f3-444ddedb5e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:08.282779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dab9d5-c623-4e66-87f3-444ddedb5e2f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:08.296005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1105/2248] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:31:08.302135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a6f8f-f7d2-4665-bfb1-a239872c12d0 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:08.314171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a6f8f-f7d2-4665-bfb1-a239872c12d0 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:08.323374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302875b-a6b7-4621-bf55-366e8ecc8625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:08.323374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302875b-a6b7-4621-bf55-366e8ecc8625 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:08.323374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302875b-a6b7-4621-bf55-366e8ecc8625 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:08.323374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302875b-a6b7-4621-bf55-366e8ecc8625 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:08.324106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302875b-a6b7-4621-bf55-366e8ecc8625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:08.326043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302875b-a6b7-4621-bf55-366e8ecc8625 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:08.339697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1144/2249] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:31:08.346443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab9d5-c623-4e66-87f3-444ddedb5e2f admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:08.358813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab9d5-c623-4e66-87f3-444ddedb5e2f admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:08.368019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc7eaf-d3cb-449e-8920-307311d4b686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:08.368019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc7eaf-d3cb-449e-8920-307311d4b686 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:08.368019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc7eaf-d3cb-449e-8920-307311d4b686 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:08.368019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc7eaf-d3cb-449e-8920-307311d4b686 admin admin] PATH_INFO: `/v3/projects/b2273d7ef30544ae88d8466c8a58cf7b/users/8486f58d7ff44b83b00408a3c0db3afe/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:08.394832 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc7eaf-d3cb-449e-8920-307311d4b686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2273d7ef30544ae88d8466c8a58cf7b, user_id=8486f58d7ff44b83b00408a3c0db3afe, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:08.400990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc7eaf-d3cb-449e-8920-307311d4b686 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:08.513164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1106/2250] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:31:08 2022] PUT /identity/v3/projects/b2273d7ef30544ae88d8466c8a58cf7b/users/8486f58d7ff44b83b00408a3c0db3afe/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:08.526348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ff860-529f-499e-94db-142fbcb49aa7 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:08.526777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ff860-529f-499e-94db-142fbcb49aa7 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:08.527208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ff860-529f-499e-94db-142fbcb49aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:08.569541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-157ff860-529f-499e-94db-142fbcb49aa7 None None] MFA Rules not processed for user `8486f58d7ff44b83b00408a3c0db3afe`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:08.571050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-157ff860-529f-499e-94db-142fbcb49aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:08.670710 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1145/2251] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:08.676157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc7eaf-d3cb-449e-8920-307311d4b686 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:08.763028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc7eaf-d3cb-449e-8920-307311d4b686 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:08.769296 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cec145-3546-42d8-b90b-17d337083ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:08.771113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cec145-3546-42d8-b90b-17d337083ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:08.771642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cec145-3546-42d8-b90b-17d337083ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:08.772033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cec145-3546-42d8-b90b-17d337083ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:08.773224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cec145-3546-42d8-b90b-17d337083ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:08.776157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cec145-3546-42d8-b90b-17d337083ba3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:08.889088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1107/2252] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:31:08 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:31:08.891980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ff860-529f-499e-94db-142fbcb49aa7 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:08.908218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ff860-529f-499e-94db-142fbcb49aa7 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:08.985839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1384a2-00f5-478f-95ec-89f3793fc474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:08.986725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1384a2-00f5-478f-95ec-89f3793fc474 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:08.987050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1384a2-00f5-478f-95ec-89f3793fc474 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:08.987363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1384a2-00f5-478f-95ec-89f3793fc474 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:08.988102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1384a2-00f5-478f-95ec-89f3793fc474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:08.990445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1384a2-00f5-478f-95ec-89f3793fc474 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:09.122645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1146/2253] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:31:08 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:31:09.129127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cec145-3546-42d8-b90b-17d337083ba3 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:09.154713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cec145-3546-42d8-b90b-17d337083ba3 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:09.161077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec203-0699-4155-85fe-2ed46b9da686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:09.161874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec203-0699-4155-85fe-2ed46b9da686 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:09.162231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec203-0699-4155-85fe-2ed46b9da686 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:09.162615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec203-0699-4155-85fe-2ed46b9da686 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:09.163326 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aec203-0699-4155-85fe-2ed46b9da686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:09.165250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aec203-0699-4155-85fe-2ed46b9da686 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:09.179792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1108/2254] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:31:09.184520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1384a2-00f5-478f-95ec-89f3793fc474 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:09.196933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1384a2-00f5-478f-95ec-89f3793fc474 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:09.202046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d697d6-a537-4f54-a7d2-74c54873aa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:09.203659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d697d6-a537-4f54-a7d2-74c54873aa1f admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:09.203999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d697d6-a537-4f54-a7d2-74c54873aa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:09.204319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d697d6-a537-4f54-a7d2-74c54873aa1f admin admin] PATH_INFO: `/v3/projects/f808a11bc5864f6bbaa9b8471225b72f/users/c4f275303e0c473db659b971d95a22a4/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:09.224314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d697d6-a537-4f54-a7d2-74c54873aa1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f808a11bc5864f6bbaa9b8471225b72f, user_id=c4f275303e0c473db659b971d95a22a4, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:09.227143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d697d6-a537-4f54-a7d2-74c54873aa1f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:09.397527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1147/2255] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:31:09 2022] PUT /identity/v3/projects/f808a11bc5864f6bbaa9b8471225b72f/users/c4f275303e0c473db659b971d95a22a4/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:09.403760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec203-0699-4155-85fe-2ed46b9da686 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:09.414697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec203-0699-4155-85fe-2ed46b9da686 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:09.443292 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:09.534825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd964e-d0a2-49d9-81fd-15c91dc0843f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:09.534825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd964e-d0a2-49d9-81fd-15c91dc0843f admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:09.534825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd964e-d0a2-49d9-81fd-15c91dc0843f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:09.534825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd964e-d0a2-49d9-81fd-15c91dc0843f admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:09.535471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd964e-d0a2-49d9-81fd-15c91dc0843f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:09.540342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd964e-d0a2-49d9-81fd-15c91dc0843f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:09.557834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1109/2256] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:31:09.562937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d697d6-a537-4f54-a7d2-74c54873aa1f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:09.574037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:09.580579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d697d6-a537-4f54-a7d2-74c54873aa1f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:09.586104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2b8a4-9f15-4f14-82a2-4da22ad7f295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:09.586939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2b8a4-9f15-4f14-82a2-4da22ad7f295 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:09.587310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2b8a4-9f15-4f14-82a2-4da22ad7f295 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:09.587740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2b8a4-9f15-4f14-82a2-4da22ad7f295 admin admin] PATH_INFO: `/v3/projects/f808a11bc5864f6bbaa9b8471225b72f/users/c4f275303e0c473db659b971d95a22a4/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:09.590542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2b8a4-9f15-4f14-82a2-4da22ad7f295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f808a11bc5864f6bbaa9b8471225b72f, user_id=c4f275303e0c473db659b971d95a22a4, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:09.594081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2b8a4-9f15-4f14-82a2-4da22ad7f295 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:09.607799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1148/2257] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:31:09 2022] PUT /identity/v3/projects/f808a11bc5864f6bbaa9b8471225b72f/users/c4f275303e0c473db659b971d95a22a4/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:31:09.619933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686f37b-900f-402c-a303-8ff98980b0c5 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:09.620258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686f37b-900f-402c-a303-8ff98980b0c5 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:09.620565 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686f37b-900f-402c-a303-8ff98980b0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:09.660287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-4686f37b-900f-402c-a303-8ff98980b0c5 None None] MFA Rules not processed for user `c4f275303e0c473db659b971d95a22a4`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:09.664127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4686f37b-900f-402c-a303-8ff98980b0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:09.754764 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1110/2258] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:31:09.775319 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2b8a4-9f15-4f14-82a2-4da22ad7f295 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:09.778246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-15a2b8a4-9f15-4f14-82a2-4da22ad7f295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:09.860394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2b8a4-9f15-4f14-82a2-4da22ad7f295 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:09.864568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad2418-ea62-4075-9ec6-8b7d0415c9ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:09.865208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad2418-ea62-4075-9ec6-8b7d0415c9ab service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:09.865487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad2418-ea62-4075-9ec6-8b7d0415c9ab service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:09.865760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad2418-ea62-4075-9ec6-8b7d0415c9ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:09.877180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad2418-ea62-4075-9ec6-8b7d0415c9ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:09.878700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad2418-ea62-4075-9ec6-8b7d0415c9ab service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:09.891279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1149/2259] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:10.247276 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686f37b-900f-402c-a303-8ff98980b0c5 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:10.259212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686f37b-900f-402c-a303-8ff98980b0c5 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:10.263396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bb028-a79c-451e-b0c9-bf9fa0e6b41d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:10.264038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bb028-a79c-451e-b0c9-bf9fa0e6b41d service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:10.264373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bb028-a79c-451e-b0c9-bf9fa0e6b41d service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:10.264587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bb028-a79c-451e-b0c9-bf9fa0e6b41d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:10.338147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bb028-a79c-451e-b0c9-bf9fa0e6b41d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:10.339978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bb028-a79c-451e-b0c9-bf9fa0e6b41d service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:10.351175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1111/2260] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:19.884796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:20.346396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=80 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:27.781863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2a8a9-6d42-47ef-a9bb-dc7fc7419119 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:27.783170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2a8a9-6d42-47ef-a9bb-dc7fc7419119 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:27.784005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2a8a9-6d42-47ef-a9bb-dc7fc7419119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:27.811133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-44e2a8a9-6d42-47ef-a9bb-dc7fc7419119 None None] MFA Rules not processed for user `b0db362075b04b2294a6268576a5bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:27.817322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-44e2a8a9-6d42-47ef-a9bb-dc7fc7419119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:27.899235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1150/2261] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:27.959872 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bb028-a79c-451e-b0c9-bf9fa0e6b41d service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:27.962170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8c7bb028-a79c-451e-b0c9-bf9fa0e6b41d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:28.072095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bb028-a79c-451e-b0c9-bf9fa0e6b41d service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:28.078342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab130c-5ad2-4092-b674-c30d19583651 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea4c2870e2754f3188228fc441950df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:28.079331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab130c-5ad2-4092-b674-c30d19583651 service glance] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:28.079736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab130c-5ad2-4092-b674-c30d19583651 service glance] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:28.080133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab130c-5ad2-4092-b674-c30d19583651 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:28.082898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d7ab130c-5ad2-4092-b674-c30d19583651 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:28.194828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab130c-5ad2-4092-b674-c30d19583651 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:28.196639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab130c-5ad2-4092-b674-c30d19583651 service glance] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:28.211855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1112/2262] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:28.853663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2a8a9-6d42-47ef-a9bb-dc7fc7419119 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:28.934980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2a8a9-6d42-47ef-a9bb-dc7fc7419119 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:28.940532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:28.941185 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:28.941463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:28.941735 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] PATH_INFO: `/v3/users/8486f58d7ff44b83b00408a3c0db3afe` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:28.943000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8486f58d7ff44b83b00408a3c0db3afe)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:28.945019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:29.227506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8486f58d7ff44b83b00408a3c0db3afe'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:29.228893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8486f58d7ff44b83b00408a3c0db3afe'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:29.230132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8486f58d7ff44b83b00408a3c0db3afe'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:29.321709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8486f58d7ff44b83b00408a3c0db3afe'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:29.338621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8486f58d7ff44b83b00408a3c0db3afe'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:29.375582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1151/2263] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:31:28 2022] DELETE /identity/v3/users/8486f58d7ff44b83b00408a3c0db3afe => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:31:29.382731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ab1b2-2f4e-4112-aa69-6777dabfa5b8 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:29.383503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ab1b2-2f4e-4112-aa69-6777dabfa5b8 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:29.383884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ab1b2-2f4e-4112-aa69-6777dabfa5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:29.408239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-df3ab1b2-2f4e-4112-aa69-6777dabfa5b8 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:29.409505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-df3ab1b2-2f4e-4112-aa69-6777dabfa5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:29.497331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1113/2264] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:31:29.509253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:29.510970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:29.544596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:29.579276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9490-8686-4926-9a52-853beca011b9 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:29.583149 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e0c9b-c481-4980-9fd5-919478bfff77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:29.583707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0c9b-c481-4980-9fd5-919478bfff77 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:29.583983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0c9b-c481-4980-9fd5-919478bfff77 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:29.584258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0c9b-c481-4980-9fd5-919478bfff77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:29.595453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e0c9b-c481-4980-9fd5-919478bfff77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:29.597231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e0c9b-c481-4980-9fd5-919478bfff77 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:29.611538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1152/2265] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:30.221823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ab1b2-2f4e-4112-aa69-6777dabfa5b8 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:30.224579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-df3ab1b2-2f4e-4112-aa69-6777dabfa5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:30.253894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ab1b2-2f4e-4112-aa69-6777dabfa5b8 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:30.263492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:30.265127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:30.265538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:30.265942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] PATH_INFO: `/v3/projects/b2273d7ef30544ae88d8466c8a58cf7b` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:30.268121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2273d7ef30544ae88d8466c8a58cf7b)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:30.271927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:30.511347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:30.523951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2273d7ef30544ae88d8466c8a58cf7b', 'request_id': 'req-28772a80-281b-4a15-a68a-28757cb0eee2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:30.531509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2273d7ef30544ae88d8466c8a58cf7b', 'request_id': 'req-28772a80-281b-4a15-a68a-28757cb0eee2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:30.612704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2273d7ef30544ae88d8466c8a58cf7b', 'request_id': 'req-28772a80-281b-4a15-a68a-28757cb0eee2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:30.632240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] The token cache is being invalidate because project b2273d7ef30544ae88d8466c8a58cf7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:31:30.632583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:30.633447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:30.635437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1114/2266] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:31:30 2022] DELETE /identity/v3/projects/b2273d7ef30544ae88d8466c8a58cf7b => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:30.641845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e0c9b-c481-4980-9fd5-919478bfff77 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:30.654139 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e0c9b-c481-4980-9fd5-919478bfff77 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:30.727573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:30.728256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:30.728642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:30.729001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] PATH_INFO: `/v3/users/c4f275303e0c473db659b971d95a22a4` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:30.730635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f275303e0c473db659b971d95a22a4)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:30.732695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:31.047613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f275303e0c473db659b971d95a22a4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:31.048471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f275303e0c473db659b971d95a22a4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:31.049460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f275303e0c473db659b971d95a22a4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:31.166741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f275303e0c473db659b971d95a22a4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:31.176980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-fad3699b-9f6e-477e-8847-c40818aab550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f275303e0c473db659b971d95a22a4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:31.198165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1153/2267] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:31:30 2022] DELETE /identity/v3/users/c4f275303e0c473db659b971d95a22a4 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:31.914196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:31.916118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:31.964598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28772a80-281b-4a15-a68a-28757cb0eee2 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:32.070799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:32.071405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:32.072092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:32.072519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] PATH_INFO: `/v3/projects/f808a11bc5864f6bbaa9b8471225b72f` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:32.074626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f808a11bc5864f6bbaa9b8471225b72f)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:32.077930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:32.226277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f808a11bc5864f6bbaa9b8471225b72f', 'request_id': 'req-ef398510-d6b0-495e-929b-47486f127d36'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:32.233954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f808a11bc5864f6bbaa9b8471225b72f', 'request_id': 'req-ef398510-d6b0-495e-929b-47486f127d36'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:32.321069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f808a11bc5864f6bbaa9b8471225b72f', 'request_id': 'req-ef398510-d6b0-495e-929b-47486f127d36'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:32.336696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] The token cache is being invalidate because project f808a11bc5864f6bbaa9b8471225b72f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:31:32.336991 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:32.337683 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ef398510-d6b0-495e-929b-47486f127d36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:32.339367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1115/2268] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:31:31 2022] DELETE /identity/v3/projects/f808a11bc5864f6bbaa9b8471225b72f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:32.345725 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268ab1a-fa29-4c8c-ae20-e63d234b6690 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:32.346331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268ab1a-fa29-4c8c-ae20-e63d234b6690 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:32.346571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268ab1a-fa29-4c8c-ae20-e63d234b6690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:32.373613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c268ab1a-fa29-4c8c-ae20-e63d234b6690 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:32.375578 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c268ab1a-fa29-4c8c-ae20-e63d234b6690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:32.433144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:32.459360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1154/2269] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:32.473792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6da12b-40ac-4e86-8b45-1747c49feb5c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:32.474303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6da12b-40ac-4e86-8b45-1747c49feb5c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:32.474737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6da12b-40ac-4e86-8b45-1747c49feb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:32.506267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-fd6da12b-40ac-4e86-8b45-1747c49feb5c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:32.507620 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fd6da12b-40ac-4e86-8b45-1747c49feb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:32.522775 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1116/2270] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:31:32.524510 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268ab1a-fa29-4c8c-ae20-e63d234b6690 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:32.537271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268ab1a-fa29-4c8c-ae20-e63d234b6690 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:32.543314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56958d9-a18f-4bf8-853c-d17451d79dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:32.544141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56958d9-a18f-4bf8-853c-d17451d79dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:32.544568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56958d9-a18f-4bf8-853c-d17451d79dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:32.544905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56958d9-a18f-4bf8-853c-d17451d79dc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:32.546557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56958d9-a18f-4bf8-853c-d17451d79dc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:31:32.547231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56958d9-a18f-4bf8-853c-d17451d79dc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:32.549293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56958d9-a18f-4bf8-853c-d17451d79dc8 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:32.566383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1155/2271] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:31:32.571520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6da12b-40ac-4e86-8b45-1747c49feb5c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:32.584429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6da12b-40ac-4e86-8b45-1747c49feb5c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:32.590016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b8dae-a191-4a4a-a5d7-278433c66925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:32.590704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b8dae-a191-4a4a-a5d7-278433c66925 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:32.590982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b8dae-a191-4a4a-a5d7-278433c66925 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:32.591525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b8dae-a191-4a4a-a5d7-278433c66925 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:32.592207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b8dae-a191-4a4a-a5d7-278433c66925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:32.595061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b8dae-a191-4a4a-a5d7-278433c66925 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:32.699469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56958d9-a18f-4bf8-853c-d17451d79dc8 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:32.703149 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c56958d9-a18f-4bf8-853c-d17451d79dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:32.727583 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1117/2272] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:31:32 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:31:32.733730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b8dae-a191-4a4a-a5d7-278433c66925 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:32.802886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:32.856206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56958d9-a18f-4bf8-853c-d17451d79dc8 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:32.870149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b8dae-a191-4a4a-a5d7-278433c66925 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:32.879187 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c2f1b-9a00-4a9b-8b81-297f31ff37a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:32.881638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c2f1b-9a00-4a9b-8b81-297f31ff37a2 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:32.882278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c2f1b-9a00-4a9b-8b81-297f31ff37a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:32.882984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c2f1b-9a00-4a9b-8b81-297f31ff37a2 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:32.884403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c2f1b-9a00-4a9b-8b81-297f31ff37a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:32.889208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c2f1b-9a00-4a9b-8b81-297f31ff37a2 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:32.959187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c7169-bd93-48c5-aa5c-f0a63a03e673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:32.960007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c7169-bd93-48c5-aa5c-f0a63a03e673 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:32.960617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c7169-bd93-48c5-aa5c-f0a63a03e673 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:32.960898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c7169-bd93-48c5-aa5c-f0a63a03e673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:32.963194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-197c7169-bd93-48c5-aa5c-f0a63a03e673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:33.066604 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1118/2273] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:31:32 2022] POST /identity/v3/users => generated 281 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:31:33.072113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c2f1b-9a00-4a9b-8b81-297f31ff37a2 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:33.072331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c7169-bd93-48c5-aa5c-f0a63a03e673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:33.075330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c7169-bd93-48c5-aa5c-f0a63a03e673 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:33.084856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c2f1b-9a00-4a9b-8b81-297f31ff37a2 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:33.093607 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b178e71-aa83-4853-9ca1-52d56275f735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:33.094511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b178e71-aa83-4853-9ca1-52d56275f735 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:33.094963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b178e71-aa83-4853-9ca1-52d56275f735 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:33.095605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b178e71-aa83-4853-9ca1-52d56275f735 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:33.096762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b178e71-aa83-4853-9ca1-52d56275f735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:33.099011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b178e71-aa83-4853-9ca1-52d56275f735 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:33.102589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1156/2274] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:31:33.111813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1119/2275] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:31:33.120758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c7169-bd93-48c5-aa5c-f0a63a03e673 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:33.133091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c7169-bd93-48c5-aa5c-f0a63a03e673 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:33.138799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a5fa6-2024-4a82-ae89-0e60e26493c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:33.139298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a5fa6-2024-4a82-ae89-0e60e26493c6 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:33.139657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a5fa6-2024-4a82-ae89-0e60e26493c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:33.139939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a5fa6-2024-4a82-ae89-0e60e26493c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:33.140607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a5fa6-2024-4a82-ae89-0e60e26493c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:33.142560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a5fa6-2024-4a82-ae89-0e60e26493c6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:33.158956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1157/2276] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:31:33.159550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b178e71-aa83-4853-9ca1-52d56275f735 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:33.173876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b178e71-aa83-4853-9ca1-52d56275f735 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:33.180679 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58567e9-08cc-4676-aa95-3855112a01bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:33.181023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58567e9-08cc-4676-aa95-3855112a01bc admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:33.181309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58567e9-08cc-4676-aa95-3855112a01bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:33.181603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58567e9-08cc-4676-aa95-3855112a01bc admin admin] PATH_INFO: `/v3/projects/879691c345da4516bf92ea719fa66dc4/users/ef49482c38dc428ab9f9a27d7178366f/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:33.208704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58567e9-08cc-4676-aa95-3855112a01bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=879691c345da4516bf92ea719fa66dc4, user_id=ef49482c38dc428ab9f9a27d7178366f, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:33.212629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58567e9-08cc-4676-aa95-3855112a01bc admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:33.301670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1120/2277] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:31:33 2022] PUT /identity/v3/projects/879691c345da4516bf92ea719fa66dc4/users/ef49482c38dc428ab9f9a27d7178366f/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:31:33.314138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487caf50-67d9-4a98-93c5-34ac1e4602f5 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:33.314409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487caf50-67d9-4a98-93c5-34ac1e4602f5 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:33.314713 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487caf50-67d9-4a98-93c5-34ac1e4602f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:33.347163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-487caf50-67d9-4a98-93c5-34ac1e4602f5 None None] MFA Rules not processed for user `ef49482c38dc428ab9f9a27d7178366f`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:33.348328 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-487caf50-67d9-4a98-93c5-34ac1e4602f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:33.361531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:33.451246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1158/2278] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:31:33.501174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58567e9-08cc-4676-aa95-3855112a01bc admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:33.503361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e58567e9-08cc-4676-aa95-3855112a01bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:33.582004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58567e9-08cc-4676-aa95-3855112a01bc admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:33.587431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8bde5-1eb4-4fe1-87cf-70376bf43a1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:33.588258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8bde5-1eb4-4fe1-87cf-70376bf43a1c service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:33.588767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8bde5-1eb4-4fe1-87cf-70376bf43a1c service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:33.589130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8bde5-1eb4-4fe1-87cf-70376bf43a1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:33.607634 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8bde5-1eb4-4fe1-87cf-70376bf43a1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:33.611389 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8bde5-1eb4-4fe1-87cf-70376bf43a1c service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:33.629215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1121/2279] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:33.649043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487caf50-67d9-4a98-93c5-34ac1e4602f5 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:33.771722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487caf50-67d9-4a98-93c5-34ac1e4602f5 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:33.777607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:33.778517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:33.779579 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:33.781807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] PATH_INFO: `/v3/users/ef49482c38dc428ab9f9a27d7178366f` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:33.782723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef49482c38dc428ab9f9a27d7178366f)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:33.784795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:34.025117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef49482c38dc428ab9f9a27d7178366f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:34.025941 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef49482c38dc428ab9f9a27d7178366f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:34.026911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef49482c38dc428ab9f9a27d7178366f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:34.143383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef49482c38dc428ab9f9a27d7178366f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:34.189201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef49482c38dc428ab9f9a27d7178366f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:34.259166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1159/2280] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:31:33 2022] DELETE /identity/v3/users/ef49482c38dc428ab9f9a27d7178366f => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:34.265226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b2b2b-692f-46aa-b7b5-6cd33afb467e None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:34.265226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b2b2b-692f-46aa-b7b5-6cd33afb467e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:34.265226 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b2b2b-692f-46aa-b7b5-6cd33afb467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:34.305664 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d36b2b2b-692f-46aa-b7b5-6cd33afb467e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:34.308164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d36b2b2b-692f-46aa-b7b5-6cd33afb467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:34.442958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1122/2281] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:34.460962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:34.466825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:34.529109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:34.569141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5ea0-71a2-4db1-af98-ddded2f16a1b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:34.573824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaff14a-7ecc-4711-b11a-8db5f5b7c42e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:34.574057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaff14a-7ecc-4711-b11a-8db5f5b7c42e service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:34.574261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaff14a-7ecc-4711-b11a-8db5f5b7c42e service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:34.574597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaff14a-7ecc-4711-b11a-8db5f5b7c42e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:34.586256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaff14a-7ecc-4711-b11a-8db5f5b7c42e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:34.587844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaff14a-7ecc-4711-b11a-8db5f5b7c42e service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:34.599404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1160/2282] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:35.250965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b2b2b-692f-46aa-b7b5-6cd33afb467e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:35.253479 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d36b2b2b-692f-46aa-b7b5-6cd33afb467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:35.287624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b2b2b-692f-46aa-b7b5-6cd33afb467e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:35.303870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:35.305118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:35.305702 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:35.306091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] PATH_INFO: `/v3/projects/879691c345da4516bf92ea719fa66dc4` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:35.308730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] RBAC: Authorizing `identity:delete_project(project_id=879691c345da4516bf92ea719fa66dc4)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:35.313240 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:35.355576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:35.467506 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879691c345da4516bf92ea719fa66dc4', 'request_id': 'req-35a42d87-b9e0-4920-8586-f0d5eb51603b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:35.481957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879691c345da4516bf92ea719fa66dc4', 'request_id': 'req-35a42d87-b9e0-4920-8586-f0d5eb51603b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:35.589614 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879691c345da4516bf92ea719fa66dc4', 'request_id': 'req-35a42d87-b9e0-4920-8586-f0d5eb51603b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:35.623970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] The token cache is being invalidate because project 879691c345da4516bf92ea719fa66dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:31:35.624690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:35.626935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-35a42d87-b9e0-4920-8586-f0d5eb51603b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:35.629862 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1123/2283] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:31:35 2022] DELETE /identity/v3/projects/879691c345da4516bf92ea719fa66dc4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:35.638255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588d345-0a53-4320-96fa-ffde22767f57 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:35.642261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588d345-0a53-4320-96fa-ffde22767f57 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:35.642261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588d345-0a53-4320-96fa-ffde22767f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:35.694513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-a588d345-0a53-4320-96fa-ffde22767f57 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:35.696569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a588d345-0a53-4320-96fa-ffde22767f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:35.822797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1161/2284] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:35.833246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707ce9e-fb37-4058-861b-d9ba5ba07066 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:35.833578 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707ce9e-fb37-4058-861b-d9ba5ba07066 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:35.833804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707ce9e-fb37-4058-861b-d9ba5ba07066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:35.856528 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-4707ce9e-fb37-4058-861b-d9ba5ba07066 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:35.857795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4707ce9e-fb37-4058-861b-d9ba5ba07066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:35.867113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1124/2285] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:35.871706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588d345-0a53-4320-96fa-ffde22767f57 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:35.883208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588d345-0a53-4320-96fa-ffde22767f57 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:35.889139 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeddc32-6944-476d-b7f8-4cbf64260eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:35.889920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeddc32-6944-476d-b7f8-4cbf64260eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:35.890231 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeddc32-6944-476d-b7f8-4cbf64260eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:35.890729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeddc32-6944-476d-b7f8-4cbf64260eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:35.891705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeddc32-6944-476d-b7f8-4cbf64260eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:31:35.892114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeddc32-6944-476d-b7f8-4cbf64260eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:35.894012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeddc32-6944-476d-b7f8-4cbf64260eb5 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:35.909837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1162/2286] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:31:35.914898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707ce9e-fb37-4058-861b-d9ba5ba07066 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:35.927775 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707ce9e-fb37-4058-861b-d9ba5ba07066 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:35.933372 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bdb7c-3d0c-497d-ab08-a3993a389350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:35.934016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bdb7c-3d0c-497d-ab08-a3993a389350 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:35.934287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bdb7c-3d0c-497d-ab08-a3993a389350 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:35.934604 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bdb7c-3d0c-497d-ab08-a3993a389350 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:35.935285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bdb7c-3d0c-497d-ab08-a3993a389350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:35.937540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bdb7c-3d0c-497d-ab08-a3993a389350 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:36.061767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1125/2287] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:31:35 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:31:36.067218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeddc32-6944-476d-b7f8-4cbf64260eb5 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:36.080427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeddc32-6944-476d-b7f8-4cbf64260eb5 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:36.149842 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431973-ddc4-4ee9-b800-3bfbbb8ef936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:36.150728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431973-ddc4-4ee9-b800-3bfbbb8ef936 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:36.151106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431973-ddc4-4ee9-b800-3bfbbb8ef936 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:36.151496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431973-ddc4-4ee9-b800-3bfbbb8ef936 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:36.152255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc431973-ddc4-4ee9-b800-3bfbbb8ef936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:36.154676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc431973-ddc4-4ee9-b800-3bfbbb8ef936 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:36.269835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1163/2288] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:31:36 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:31:36.274446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bdb7c-3d0c-497d-ab08-a3993a389350 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:36.287231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bdb7c-3d0c-497d-ab08-a3993a389350 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:36.292329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9300e90-49a9-4539-b873-59791777bdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:36.293388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9300e90-49a9-4539-b873-59791777bdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:36.293388 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9300e90-49a9-4539-b873-59791777bdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:36.293682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9300e90-49a9-4539-b873-59791777bdf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:36.294181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9300e90-49a9-4539-b873-59791777bdf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:36.296550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9300e90-49a9-4539-b873-59791777bdf8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:36.310127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1126/2289] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:31:36.316059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431973-ddc4-4ee9-b800-3bfbbb8ef936 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:36.327491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431973-ddc4-4ee9-b800-3bfbbb8ef936 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:36.332906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457bcd3-f720-4ab1-841a-93a7049756d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:36.334348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457bcd3-f720-4ab1-841a-93a7049756d2 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:36.334348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457bcd3-f720-4ab1-841a-93a7049756d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:36.334348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457bcd3-f720-4ab1-841a-93a7049756d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:36.334807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457bcd3-f720-4ab1-841a-93a7049756d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:36.336526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457bcd3-f720-4ab1-841a-93a7049756d2 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:36.348987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1164/2290] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:31:36.355663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9300e90-49a9-4539-b873-59791777bdf8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:36.370537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9300e90-49a9-4539-b873-59791777bdf8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:36.376833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158a3cc-3f39-45d9-bb6e-be4194930677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:36.377627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158a3cc-3f39-45d9-bb6e-be4194930677 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:36.377948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158a3cc-3f39-45d9-bb6e-be4194930677 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:36.378265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158a3cc-3f39-45d9-bb6e-be4194930677 admin admin] PATH_INFO: `/v3/projects/7508a86facdb4ff98b171aa75962dde5/users/2351ea9262c44d74827e79231d1813c6/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:36.397907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158a3cc-3f39-45d9-bb6e-be4194930677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7508a86facdb4ff98b171aa75962dde5, user_id=2351ea9262c44d74827e79231d1813c6, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:36.400921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158a3cc-3f39-45d9-bb6e-be4194930677 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:36.522943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1127/2291] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:31:36 2022] PUT /identity/v3/projects/7508a86facdb4ff98b171aa75962dde5/users/2351ea9262c44d74827e79231d1813c6/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:36.530870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b85a-8589-4adc-ba24-08f14f917f54 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:36.531353 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b85a-8589-4adc-ba24-08f14f917f54 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:36.531847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b85a-8589-4adc-ba24-08f14f917f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:36.559153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-ebf2b85a-8589-4adc-ba24-08f14f917f54 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:36.561994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ebf2b85a-8589-4adc-ba24-08f14f917f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:36.614743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:36.647568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1165/2292] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:36.659203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158a3cc-3f39-45d9-bb6e-be4194930677 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:36.661154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e158a3cc-3f39-45d9-bb6e-be4194930677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:36.736417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158a3cc-3f39-45d9-bb6e-be4194930677 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:36.741360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2138c-1717-40ec-933e-d6d8a7af234a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:36.741999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2138c-1717-40ec-933e-d6d8a7af234a service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:36.742272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2138c-1717-40ec-933e-d6d8a7af234a service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:36.742563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2138c-1717-40ec-933e-d6d8a7af234a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:36.753721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2138c-1717-40ec-933e-d6d8a7af234a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:36.755241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2138c-1717-40ec-933e-d6d8a7af234a service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:36.766805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1128/2293] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:36.817111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2b85a-8589-4adc-ba24-08f14f917f54 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:36.818541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ebf2b85a-8589-4adc-ba24-08f14f917f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:36.920772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2b85a-8589-4adc-ba24-08f14f917f54 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:36.932447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50b57f-7d4f-4f0f-8774-7bb1a8109f6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:36.934878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b57f-7d4f-4f0f-8774-7bb1a8109f6b service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:36.935551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b57f-7d4f-4f0f-8774-7bb1a8109f6b service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:36.937299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b57f-7d4f-4f0f-8774-7bb1a8109f6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:36.944514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ea50b57f-7d4f-4f0f-8774-7bb1a8109f6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:37.095437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50b57f-7d4f-4f0f-8774-7bb1a8109f6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:37.099451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50b57f-7d4f-4f0f-8774-7bb1a8109f6b service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:37.108670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1166/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:38.670221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399c6d8-ebda-4916-88ad-e4dd68b8602b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:38.672023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399c6d8-ebda-4916-88ad-e4dd68b8602b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:38.672023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399c6d8-ebda-4916-88ad-e4dd68b8602b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:38.708451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b399c6d8-ebda-4916-88ad-e4dd68b8602b None None] MFA Rules not processed for user `2351ea9262c44d74827e79231d1813c6`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:38.709807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b399c6d8-ebda-4916-88ad-e4dd68b8602b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:38.759084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:38.797881 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1129/2295] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:38.811638 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50b57f-7d4f-4f0f-8774-7bb1a8109f6b service nova] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:38.814933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ea50b57f-7d4f-4f0f-8774-7bb1a8109f6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:38.934306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50b57f-7d4f-4f0f-8774-7bb1a8109f6b service nova] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:38.941552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24082b26-f336-4be6-937d-14cb827090fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:38.942360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24082b26-f336-4be6-937d-14cb827090fd service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:38.942774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24082b26-f336-4be6-937d-14cb827090fd service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:38.943267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24082b26-f336-4be6-937d-14cb827090fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:38.957142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24082b26-f336-4be6-937d-14cb827090fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:38.963829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24082b26-f336-4be6-937d-14cb827090fd service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:38.969873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1167/2296] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:41.910934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b248-5549-4d47-a3d7-e6316316c077 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:41.911573 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b248-5549-4d47-a3d7-e6316316c077 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:41.911909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b248-5549-4d47-a3d7-e6316316c077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:41.944623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3683b248-5549-4d47-a3d7-e6316316c077 None None] MFA Rules not processed for user `2351ea9262c44d74827e79231d1813c6`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:41.946433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3683b248-5549-4d47-a3d7-e6316316c077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:41.960147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1130/2297] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:44.387754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24082b26-f336-4be6-937d-14cb827090fd service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:44.389364 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-24082b26-f336-4be6-937d-14cb827090fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:44.407486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24082b26-f336-4be6-937d-14cb827090fd service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:44.413151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:44.413886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:44.414236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:44.414608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] PATH_INFO: `/v3/users/8e045711ea2742c498e0bcd4017b52cd` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:44.415947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e045711ea2742c498e0bcd4017b52cd)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:44.418227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:44.435771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:44.685587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e045711ea2742c498e0bcd4017b52cd'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:44.686440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e045711ea2742c498e0bcd4017b52cd'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:44.687085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e045711ea2742c498e0bcd4017b52cd'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:44.795149 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e045711ea2742c498e0bcd4017b52cd'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:44.806679 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e045711ea2742c498e0bcd4017b52cd'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:44.838187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1168/2298] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:31:44 2022] DELETE /identity/v3/users/8e045711ea2742c498e0bcd4017b52cd => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:31:44.848475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e1c29-8088-4bb4-a9b0-195e295a20c5 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:44.849338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e1c29-8088-4bb4-a9b0-195e295a20c5 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:44.849498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e1c29-8088-4bb4-a9b0-195e295a20c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:44.873727 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-c33e1c29-8088-4bb4-a9b0-195e295a20c5 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:44.875098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c33e1c29-8088-4bb4-a9b0-195e295a20c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:44.956641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1131/2299] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:44.968481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:44.970227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:45.060205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1152bf-144f-4ff9-bfd4-11dd61553024 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:45.064802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c860fe-caea-490f-8ef2-2de7c4a22422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:45.065622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c860fe-caea-490f-8ef2-2de7c4a22422 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:45.066024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c860fe-caea-490f-8ef2-2de7c4a22422 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:45.066471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c860fe-caea-490f-8ef2-2de7c4a22422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:45.077172 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c860fe-caea-490f-8ef2-2de7c4a22422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:45.078676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c860fe-caea-490f-8ef2-2de7c4a22422 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:45.090634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1169/2300] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:45.674873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e1c29-8088-4bb4-a9b0-195e295a20c5 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:45.676728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c33e1c29-8088-4bb4-a9b0-195e295a20c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:45.694205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e1c29-8088-4bb4-a9b0-195e295a20c5 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:45.700514 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:45.701356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:45.701830 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:45.702309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] PATH_INFO: `/v3/projects/658ec261e291457ca3bbf1f62c9ba551` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:45.704534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658ec261e291457ca3bbf1f62c9ba551)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:45.710326 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:45.744173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c860fe-caea-490f-8ef2-2de7c4a22422 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:45.746157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-35c860fe-caea-490f-8ef2-2de7c4a22422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:45.858617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658ec261e291457ca3bbf1f62c9ba551', 'request_id': 'req-799d738b-fb6c-40a8-8845-86b2a817a608'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:45.863966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c860fe-caea-490f-8ef2-2de7c4a22422 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:45.871040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658ec261e291457ca3bbf1f62c9ba551', 'request_id': 'req-799d738b-fb6c-40a8-8845-86b2a817a608'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:45.901882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:45.956144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b4e72-f927-4077-82e4-50bc22ed05d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:45.956837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4e72-f927-4077-82e4-50bc22ed05d0 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:45.957170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4e72-f927-4077-82e4-50bc22ed05d0 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:45.957520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4e72-f927-4077-82e4-50bc22ed05d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:45.959557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fa7b4e72-f927-4077-82e4-50bc22ed05d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:46.029046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b4e72-f927-4077-82e4-50bc22ed05d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:46.031468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b4e72-f927-4077-82e4-50bc22ed05d0 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:46.042672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1170/2301] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:31:46.045174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658ec261e291457ca3bbf1f62c9ba551', 'request_id': 'req-799d738b-fb6c-40a8-8845-86b2a817a608'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:46.066278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] The token cache is being invalidate because project 658ec261e291457ca3bbf1f62c9ba551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:31:46.067428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:46.068685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-799d738b-fb6c-40a8-8845-86b2a817a608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:31:46.071837 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1132/2302] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:31:45 2022] DELETE /identity/v3/projects/658ec261e291457ca3bbf1f62c9ba551 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:46.079959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a64fdd-9c66-42f7-a7e1-58f247d0a11d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:46.080578 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a64fdd-9c66-42f7-a7e1-58f247d0a11d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:46.080928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a64fdd-9c66-42f7-a7e1-58f247d0a11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:46.118559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d1a64fdd-9c66-42f7-a7e1-58f247d0a11d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:46.122542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d1a64fdd-9c66-42f7-a7e1-58f247d0a11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:46.264014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1171/2303] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:46.274789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e59c5e-030f-4989-a6bc-23ff0d257bd4 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:46.275607 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e59c5e-030f-4989-a6bc-23ff0d257bd4 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:46.276066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e59c5e-030f-4989-a6bc-23ff0d257bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:46.317787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-30e59c5e-030f-4989-a6bc-23ff0d257bd4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:46.321543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-30e59c5e-030f-4989-a6bc-23ff0d257bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:46.336922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1133/2304] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:46.344642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a64fdd-9c66-42f7-a7e1-58f247d0a11d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:46.370567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a64fdd-9c66-42f7-a7e1-58f247d0a11d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:46.381629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd93fb-f2b4-41ba-8e82-37b1953e11dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:46.382741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd93fb-f2b4-41ba-8e82-37b1953e11dc admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:46.383238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd93fb-f2b4-41ba-8e82-37b1953e11dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:46.383807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd93fb-f2b4-41ba-8e82-37b1953e11dc admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:46.386671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd93fb-f2b4-41ba-8e82-37b1953e11dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:31:46.387770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd93fb-f2b4-41ba-8e82-37b1953e11dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:46.392751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd93fb-f2b4-41ba-8e82-37b1953e11dc admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:46.418593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1172/2305] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:31:46.422395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e59c5e-030f-4989-a6bc-23ff0d257bd4 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:46.437966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e59c5e-030f-4989-a6bc-23ff0d257bd4 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:46.450360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1bd57f-3ba3-4d2f-a2b5-d120441a3813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:46.450360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bd57f-3ba3-4d2f-a2b5-d120441a3813 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:46.450360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bd57f-3ba3-4d2f-a2b5-d120441a3813 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:46.450360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bd57f-3ba3-4d2f-a2b5-d120441a3813 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:46.451149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1bd57f-3ba3-4d2f-a2b5-d120441a3813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:46.455470 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1bd57f-3ba3-4d2f-a2b5-d120441a3813 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:46.584936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1134/2306] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:31:46 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:31:46.590831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd93fb-f2b4-41ba-8e82-37b1953e11dc admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:46.614544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd93fb-f2b4-41ba-8e82-37b1953e11dc admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:46.728296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:46.741376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb38782-a789-449b-b0a3-479fc17c6afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:46.742756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb38782-a789-449b-b0a3-479fc17c6afb admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:46.742971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb38782-a789-449b-b0a3-479fc17c6afb admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:46.743473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb38782-a789-449b-b0a3-479fc17c6afb admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:46.744477 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb38782-a789-449b-b0a3-479fc17c6afb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:46.748616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb38782-a789-449b-b0a3-479fc17c6afb admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:46.875158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1173/2307] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:31:46 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:31:46.880544 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1bd57f-3ba3-4d2f-a2b5-d120441a3813 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:46.897319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1bd57f-3ba3-4d2f-a2b5-d120441a3813 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:46.902893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0e8ce-797d-44dc-a787-8910d92ffcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:46.903844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0e8ce-797d-44dc-a787-8910d92ffcc5 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:46.904284 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0e8ce-797d-44dc-a787-8910d92ffcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:46.905558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0e8ce-797d-44dc-a787-8910d92ffcc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:46.906608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0e8ce-797d-44dc-a787-8910d92ffcc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:46.908778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0e8ce-797d-44dc-a787-8910d92ffcc5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:46.931839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1135/2308] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:31:46.941255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb38782-a789-449b-b0a3-479fc17c6afb admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:46.962612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb38782-a789-449b-b0a3-479fc17c6afb admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:46.972010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6c3ad-85ed-484c-9bc9-0efaf6e804b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:46.973073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6c3ad-85ed-484c-9bc9-0efaf6e804b3 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:46.973554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6c3ad-85ed-484c-9bc9-0efaf6e804b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:46.974032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6c3ad-85ed-484c-9bc9-0efaf6e804b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:46.975355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6c3ad-85ed-484c-9bc9-0efaf6e804b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:46.978682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6c3ad-85ed-484c-9bc9-0efaf6e804b3 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:46.996982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1174/2309] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:31:47.002535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0e8ce-797d-44dc-a787-8910d92ffcc5 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:47.022070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0e8ce-797d-44dc-a787-8910d92ffcc5 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:47.029368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a41061-099c-4c2a-966a-ec7a01fe9f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:47.034417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a41061-099c-4c2a-966a-ec7a01fe9f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:47.034417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a41061-099c-4c2a-966a-ec7a01fe9f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:47.034417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a41061-099c-4c2a-966a-ec7a01fe9f2f admin admin] PATH_INFO: `/v3/projects/11871f979145455499b30912da1abcfd/users/aff4cd6b5ec14cb29813416b1d929333/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:47.046960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:47.051852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a41061-099c-4c2a-966a-ec7a01fe9f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11871f979145455499b30912da1abcfd, user_id=aff4cd6b5ec14cb29813416b1d929333, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:47.054919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a41061-099c-4c2a-966a-ec7a01fe9f2f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:47.188102 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1136/2310] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:31:47 2022] PUT /identity/v3/projects/11871f979145455499b30912da1abcfd/users/aff4cd6b5ec14cb29813416b1d929333/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:31:47.196404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a3c31-6f8e-41f1-a3d6-530ea75c7d6c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:47.197000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a3c31-6f8e-41f1-a3d6-530ea75c7d6c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:47.197479 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a3c31-6f8e-41f1-a3d6-530ea75c7d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:47.230068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-3e0a3c31-6f8e-41f1-a3d6-530ea75c7d6c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:47.232069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3e0a3c31-6f8e-41f1-a3d6-530ea75c7d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:47.362653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1175/2311] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:47.375434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a41061-099c-4c2a-966a-ec7a01fe9f2f admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:47.378204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-40a41061-099c-4c2a-966a-ec7a01fe9f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:47.475070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a41061-099c-4c2a-966a-ec7a01fe9f2f admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:47.479705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6864ff-70c5-4662-a09d-99865b06f35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:47.480437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6864ff-70c5-4662-a09d-99865b06f35d service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:47.480777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6864ff-70c5-4662-a09d-99865b06f35d service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:47.481123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6864ff-70c5-4662-a09d-99865b06f35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:47.495323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6864ff-70c5-4662-a09d-99865b06f35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:47.497197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6864ff-70c5-4662-a09d-99865b06f35d service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:47.509967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1137/2312] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:31:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:31:48.713618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd5afa-9a19-4f13-8175-542116c4f94a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:48.714428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd5afa-9a19-4f13-8175-542116c4f94a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:48.714839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd5afa-9a19-4f13-8175-542116c4f94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:48.785375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-ecdd5afa-9a19-4f13-8175-542116c4f94a None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:48.788401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ecdd5afa-9a19-4f13-8175-542116c4f94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:49.034681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1176/2313] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:31:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:31:49.094196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6864ff-70c5-4662-a09d-99865b06f35d service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:49.096732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2d6864ff-70c5-4662-a09d-99865b06f35d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:49.264716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6864ff-70c5-4662-a09d-99865b06f35d service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:49.275691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b644e71-f126-4c88-925b-749bdbdfa4cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:49.278458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b644e71-f126-4c88-925b-749bdbdfa4cc service swift] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:49.278825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b644e71-f126-4c88-925b-749bdbdfa4cc service swift] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:49.279346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b644e71-f126-4c88-925b-749bdbdfa4cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:49.304391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b644e71-f126-4c88-925b-749bdbdfa4cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:49.307841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b644e71-f126-4c88-925b-749bdbdfa4cc service swift] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:49.327730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1138/2314] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:31:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:31:58.941099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:59.272829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a142bc-0e3f-40dc-911c-7d7c5ce0b3f0 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:59.273204 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a142bc-0e3f-40dc-911c-7d7c5ce0b3f0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:59.273436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a142bc-0e3f-40dc-911c-7d7c5ce0b3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:59.306885 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c7a142bc-0e3f-40dc-911c-7d7c5ce0b3f0 None None] MFA Rules not processed for user `aff4cd6b5ec14cb29813416b1d929333`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:31:59.308114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c7a142bc-0e3f-40dc-911c-7d7c5ce0b3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:31:59.320924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=62,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:31:59.492498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1177/2315] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:31:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:31:59.497420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b644e71-f126-4c88-925b-749bdbdfa4cc service swift] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:59.525179 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b644e71-f126-4c88-925b-749bdbdfa4cc service swift] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:59.536060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd64e5-7744-477c-b94b-98800c2c2b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:59.537009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd64e5-7744-477c-b94b-98800c2c2b13 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:59.541630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd64e5-7744-477c-b94b-98800c2c2b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:59.541630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd64e5-7744-477c-b94b-98800c2c2b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:59.541630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd64e5-7744-477c-b94b-98800c2c2b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:59.542956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd64e5-7744-477c-b94b-98800c2c2b13 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:31:59.666845 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1139/2316] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:31:59 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:31:59.672574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a142bc-0e3f-40dc-911c-7d7c5ce0b3f0 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:31:59.699823 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a142bc-0e3f-40dc-911c-7d7c5ce0b3f0 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:31:59.908407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75393ff8-5141-4b4f-b870-ad4dba613c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:31:59.910038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75393ff8-5141-4b4f-b870-ad4dba613c7f admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:31:59.910820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75393ff8-5141-4b4f-b870-ad4dba613c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:31:59.918655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75393ff8-5141-4b4f-b870-ad4dba613c7f admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:31:59.929931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75393ff8-5141-4b4f-b870-ad4dba613c7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:31:59.935057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75393ff8-5141-4b4f-b870-ad4dba613c7f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:00.054542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1178/2317] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:31:59 2022] POST /identity/v3/users => generated 299 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:32:00.057818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd64e5-7744-477c-b94b-98800c2c2b13 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:00.083737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd64e5-7744-477c-b94b-98800c2c2b13 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:00.097673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2123f-28c9-44db-abd1-bd5ecad89b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:00.098696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2123f-28c9-44db-abd1-bd5ecad89b05 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:00.099152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2123f-28c9-44db-abd1-bd5ecad89b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:00.099594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2123f-28c9-44db-abd1-bd5ecad89b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:00.100712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2123f-28c9-44db-abd1-bd5ecad89b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:00.104458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2123f-28c9-44db-abd1-bd5ecad89b05 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:00.136414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1140/2318] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:32:00.138454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75393ff8-5141-4b4f-b870-ad4dba613c7f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:00.166054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75393ff8-5141-4b4f-b870-ad4dba613c7f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:00.176437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f42fd1-a36d-4690-a3c1-f02bd6ddbc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:00.177962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f42fd1-a36d-4690-a3c1-f02bd6ddbc5b admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:00.178536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f42fd1-a36d-4690-a3c1-f02bd6ddbc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:00.179695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f42fd1-a36d-4690-a3c1-f02bd6ddbc5b admin admin] PATH_INFO: `/v3/projects/d275a9099ec54100956f75274cd16bbb/users/192fbd27593e434591a091944ccfcfc5/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:00.240524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f42fd1-a36d-4690-a3c1-f02bd6ddbc5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d275a9099ec54100956f75274cd16bbb, user_id=192fbd27593e434591a091944ccfcfc5, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:00.246070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f42fd1-a36d-4690-a3c1-f02bd6ddbc5b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:00.337808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1179/2319] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:32:00 2022] PUT /identity/v3/projects/d275a9099ec54100956f75274cd16bbb/users/192fbd27593e434591a091944ccfcfc5/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:32:00.347796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2123f-28c9-44db-abd1-bd5ecad89b05 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:00.391503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2123f-28c9-44db-abd1-bd5ecad89b05 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:00.614330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1887547-f473-4646-8b64-ac7ecf6893fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:00.616021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1887547-f473-4646-8b64-ac7ecf6893fe admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:00.616916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1887547-f473-4646-8b64-ac7ecf6893fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:00.626373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1887547-f473-4646-8b64-ac7ecf6893fe admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:00.628578 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1887547-f473-4646-8b64-ac7ecf6893fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:00.632974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1887547-f473-4646-8b64-ac7ecf6893fe admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:00.658822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1141/2320] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:32:00.665274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f42fd1-a36d-4690-a3c1-f02bd6ddbc5b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:00.688551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f42fd1-a36d-4690-a3c1-f02bd6ddbc5b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:00.698577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe5ae7-97cb-4c4f-9866-579d51b7ae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:00.699746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe5ae7-97cb-4c4f-9866-579d51b7ae77 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:00.700272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe5ae7-97cb-4c4f-9866-579d51b7ae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:00.700784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe5ae7-97cb-4c4f-9866-579d51b7ae77 admin admin] PATH_INFO: `/v3/projects/d275a9099ec54100956f75274cd16bbb/users/192fbd27593e434591a091944ccfcfc5/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:00.705338 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe5ae7-97cb-4c4f-9866-579d51b7ae77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d275a9099ec54100956f75274cd16bbb, user_id=192fbd27593e434591a091944ccfcfc5, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:00.709927 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe5ae7-97cb-4c4f-9866-579d51b7ae77 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:00.744846 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1180/2321] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:32:00 2022] PUT /identity/v3/projects/d275a9099ec54100956f75274cd16bbb/users/192fbd27593e434591a091944ccfcfc5/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:32:10.647329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:32:10.731100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:32:12.535753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7305bf5-4599-4806-8677-27dfad2f47dc None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:12.537056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7305bf5-4599-4806-8677-27dfad2f47dc None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:12.537598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7305bf5-4599-4806-8677-27dfad2f47dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:12.561614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe5ae7-97cb-4c4f-9866-579d51b7ae77 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:12.591152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-e7305bf5-4599-4806-8677-27dfad2f47dc None None] MFA Rules not processed for user `192fbd27593e434591a091944ccfcfc5`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:32:12.609431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e7305bf5-4599-4806-8677-27dfad2f47dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:12.675716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe5ae7-97cb-4c4f-9866-579d51b7ae77 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:12.692682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f97102-fe7f-4c60-8600-04149b8dad47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:12.693479 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f97102-fe7f-4c60-8600-04149b8dad47 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:12.693876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f97102-fe7f-4c60-8600-04149b8dad47 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:12.694273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f97102-fe7f-4c60-8600-04149b8dad47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:12.697089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a8f97102-fe7f-4c60-8600-04149b8dad47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:12.717931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1142/2322] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:32:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:32:12.718627 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f97102-fe7f-4c60-8600-04149b8dad47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:12.719782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a86801-4380-40c0-be1d-c5e5ec4166a1 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:12.720449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a86801-4380-40c0-be1d-c5e5ec4166a1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:12.720449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a86801-4380-40c0-be1d-c5e5ec4166a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:12.722060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f97102-fe7f-4c60-8600-04149b8dad47 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:12.742294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1181/2323] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:32:12.743272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f97102-fe7f-4c60-8600-04149b8dad47 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:12.745487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-a8f97102-fe7f-4c60-8600-04149b8dad47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:12.759681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-83a86801-4380-40c0-be1d-c5e5ec4166a1 None None] MFA Rules not processed for user `2351ea9262c44d74827e79231d1813c6`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:32:12.761375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-83a86801-4380-40c0-be1d-c5e5ec4166a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:12.843384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1143/2324] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:32:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:32:12.882473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f97102-fe7f-4c60-8600-04149b8dad47 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:12.887198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a78987-2fa7-47ca-8a3c-d4075eb75793 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea4c2870e2754f3188228fc441950df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:12.887989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a78987-2fa7-47ca-8a3c-d4075eb75793 service glance] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:12.889163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a78987-2fa7-47ca-8a3c-d4075eb75793 service glance] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:12.889541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a78987-2fa7-47ca-8a3c-d4075eb75793 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:12.982740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a78987-2fa7-47ca-8a3c-d4075eb75793 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:12.984318 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a78987-2fa7-47ca-8a3c-d4075eb75793 service glance] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:12.995825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1182/2325] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:32:13.237115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a86801-4380-40c0-be1d-c5e5ec4166a1 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:13.240077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-83a86801-4380-40c0-be1d-c5e5ec4166a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:13.267019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a86801-4380-40c0-be1d-c5e5ec4166a1 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:13.273297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7c4fd-08db-4a0f-9539-289004dad00c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:13.274422 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7c4fd-08db-4a0f-9539-289004dad00c service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:13.274864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7c4fd-08db-4a0f-9539-289004dad00c service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:13.275332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7c4fd-08db-4a0f-9539-289004dad00c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:13.278602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f1b7c4fd-08db-4a0f-9539-289004dad00c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:13.296515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7c4fd-08db-4a0f-9539-289004dad00c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:13.298039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7c4fd-08db-4a0f-9539-289004dad00c service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:13.307951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1144/2326] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:32:13.434812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f146c86-17a2-4111-a23a-078f8d662ef5 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:13.435484 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f146c86-17a2-4111-a23a-078f8d662ef5 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:13.436406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f146c86-17a2-4111-a23a-078f8d662ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:13.473841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8f146c86-17a2-4111-a23a-078f8d662ef5 None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:32:13.475986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8f146c86-17a2-4111-a23a-078f8d662ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:13.544175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7c4fd-08db-4a0f-9539-289004dad00c service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:13.548038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f1b7c4fd-08db-4a0f-9539-289004dad00c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:13.629755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1183/2327] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:32:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:32:13.651249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f146c86-17a2-4111-a23a-078f8d662ef5 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:13.654003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7c4fd-08db-4a0f-9539-289004dad00c service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:13.655771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:32:13.661781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb079126-e27e-4861-9b13-2f085153a8f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:13.662850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb079126-e27e-4861-9b13-2f085153a8f9 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:13.663029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb079126-e27e-4861-9b13-2f085153a8f9 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:13.665317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb079126-e27e-4861-9b13-2f085153a8f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:13.666582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fb079126-e27e-4861-9b13-2f085153a8f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:13.689737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:32:13.816799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb079126-e27e-4861-9b13-2f085153a8f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:13.818185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb079126-e27e-4861-9b13-2f085153a8f9 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:13.834779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1145/2328] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:32:13.848130 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f146c86-17a2-4111-a23a-078f8d662ef5 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:13.860334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79281bc6-ee6e-464b-8563-54c7a4f343d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:13.861697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79281bc6-ee6e-464b-8563-54c7a4f343d9 service swift] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:13.862362 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79281bc6-ee6e-464b-8563-54c7a4f343d9 service swift] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:13.862948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79281bc6-ee6e-464b-8563-54c7a4f343d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:13.883308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79281bc6-ee6e-464b-8563-54c7a4f343d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:13.893821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79281bc6-ee6e-464b-8563-54c7a4f343d9 service swift] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:13.936861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1184/2329] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:32:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 17:32:13.977082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb079126-e27e-4861-9b13-2f085153a8f9 service nova] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:13.980405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-fb079126-e27e-4861-9b13-2f085153a8f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:14.126757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb079126-e27e-4861-9b13-2f085153a8f9 service nova] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:14.137542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5a892-91bf-47ce-98d0-c5e175bbd9a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:14.138953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a892-91bf-47ce-98d0-c5e175bbd9a6 service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:14.140070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a892-91bf-47ce-98d0-c5e175bbd9a6 service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:14.140653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a892-91bf-47ce-98d0-c5e175bbd9a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:14.145941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-93b5a892-91bf-47ce-98d0-c5e175bbd9a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:14.196611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5a892-91bf-47ce-98d0-c5e175bbd9a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:14.228062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5a892-91bf-47ce-98d0-c5e175bbd9a6 service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:14.299780 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1146/2330] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:32:20.989937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba43cb-0b2c-419c-ab6b-d2e4010666f4 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:20.990912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba43cb-0b2c-419c-ab6b-d2e4010666f4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:20.990912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba43cb-0b2c-419c-ab6b-d2e4010666f4 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:20.992094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1185/2331] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:32:21.001100 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785f15e-e5d1-4e3a-8ab2-82a9c83c9529 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:21.001750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785f15e-e5d1-4e3a-8ab2-82a9c83c9529 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:21.002639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785f15e-e5d1-4e3a-8ab2-82a9c83c9529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:21.064944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3785f15e-e5d1-4e3a-8ab2-82a9c83c9529 None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:32:21.070355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3785f15e-e5d1-4e3a-8ab2-82a9c83c9529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:21.104720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1147/2332] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:32:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:32:21.143748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba43cb-0b2c-419c-ab6b-d2e4010666f4 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:21.170377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba43cb-0b2c-419c-ab6b-d2e4010666f4 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:21.175116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e87c9a-c29d-4156-b637-3cc2cea7f9dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea4c2870e2754f3188228fc441950df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:21.176097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e87c9a-c29d-4156-b637-3cc2cea7f9dc service glance] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:21.176583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e87c9a-c29d-4156-b637-3cc2cea7f9dc service glance] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:21.177138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e87c9a-c29d-4156-b637-3cc2cea7f9dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:21.194720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e87c9a-c29d-4156-b637-3cc2cea7f9dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:21.196221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e87c9a-c29d-4156-b637-3cc2cea7f9dc service glance] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:21.219059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1186/2333] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:32:21.501534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeea858-d664-40e3-98e8-e5fa40434f29 None None] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:21.502203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeea858-d664-40e3-98e8-e5fa40434f29 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:21.503031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeea858-d664-40e3-98e8-e5fa40434f29 None None] PATH_INFO: `/` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:21.505243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1148/2334] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:32:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:32:21.517610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d52da-4cd1-4d96-b9d1-5753535eb2df None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:21.518098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d52da-4cd1-4d96-b9d1-5753535eb2df None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:21.518696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d52da-4cd1-4d96-b9d1-5753535eb2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:21.584179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c26d52da-4cd1-4d96-b9d1-5753535eb2df None None] MFA Rules not processed for user `c29a91e099ba4cd5886fd910d978fb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:32:21.589437 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c26d52da-4cd1-4d96-b9d1-5753535eb2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:21.604098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1187/2335] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:32:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:32:21.618923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeea858-d664-40e3-98e8-e5fa40434f29 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:21.643114 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeea858-d664-40e3-98e8-e5fa40434f29 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:21.650127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314f887-d022-468a-a2a6-d76d1c81ca80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:21.651347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314f887-d022-468a-a2a6-d76d1c81ca80 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:21.651867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314f887-d022-468a-a2a6-d76d1c81ca80 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:21.652369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314f887-d022-468a-a2a6-d76d1c81ca80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:21.663598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314f887-d022-468a-a2a6-d76d1c81ca80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:21.665531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314f887-d022-468a-a2a6-d76d1c81ca80 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:21.692531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1149/2336] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:32:21.793856 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447e2f3-0176-43a2-9749-486316866e06 None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:21.807082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447e2f3-0176-43a2-9749-486316866e06 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:21.807082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447e2f3-0176-43a2-9749-486316866e06 None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:21.807082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1188/2337] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:32:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:32:21.807416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d775b0b-d284-4385-9184-fa608729d41b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:21.807416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d775b0b-d284-4385-9184-fa608729d41b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:21.807416 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d775b0b-d284-4385-9184-fa608729d41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:21.839350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-0d775b0b-d284-4385-9184-fa608729d41b None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:32:21.842769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0d775b0b-d284-4385-9184-fa608729d41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:21.854047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1150/2338] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:32:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:32:21.872066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447e2f3-0176-43a2-9749-486316866e06 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:21.896739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447e2f3-0176-43a2-9749-486316866e06 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:21.908690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0d019-bfe4-4e51-affc-77a80a3ac555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:21.911148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0d019-bfe4-4e51-affc-77a80a3ac555 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:21.911620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0d019-bfe4-4e51-affc-77a80a3ac555 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:21.912101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0d019-bfe4-4e51-affc-77a80a3ac555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:21.941003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0d019-bfe4-4e51-affc-77a80a3ac555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:21.945083 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0d019-bfe4-4e51-affc-77a80a3ac555 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:21.969687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1189/2339] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:32:22.387625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d775b0b-d284-4385-9184-fa608729d41b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:22.390823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0d775b0b-d284-4385-9184-fa608729d41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:22.423794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d775b0b-d284-4385-9184-fa608729d41b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:22.437650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcddd7-267e-46bc-9d2c-d65bb6db7b61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:22.438381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcddd7-267e-46bc-9d2c-d65bb6db7b61 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:22.438693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcddd7-267e-46bc-9d2c-d65bb6db7b61 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:22.438968 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcddd7-267e-46bc-9d2c-d65bb6db7b61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:22.441828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-23bcddd7-267e-46bc-9d2c-d65bb6db7b61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:22.468315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcddd7-267e-46bc-9d2c-d65bb6db7b61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:22.470321 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcddd7-267e-46bc-9d2c-d65bb6db7b61 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:22.514685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1151/2340] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:32:30.274920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6d0c4-cff5-45d1-a83f-ce6fcd3bee6e None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:30.275886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6d0c4-cff5-45d1-a83f-ce6fcd3bee6e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:30.275886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6d0c4-cff5-45d1-a83f-ce6fcd3bee6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:30.312128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2ce6d0c4-cff5-45d1-a83f-ce6fcd3bee6e None None] MFA Rules not processed for user `aff4cd6b5ec14cb29813416b1d929333`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:32:30.315210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2ce6d0c4-cff5-45d1-a83f-ce6fcd3bee6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:30.334388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1190/2341] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:32:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:32:30.758121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcddd7-267e-46bc-9d2c-d65bb6db7b61 service nova] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:30.773454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcddd7-267e-46bc-9d2c-d65bb6db7b61 service nova] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:30.779948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0397b-d575-4356-82ce-97e02d28b294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:30.780756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0397b-d575-4356-82ce-97e02d28b294 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:30.781209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0397b-d575-4356-82ce-97e02d28b294 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:30.781861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0397b-d575-4356-82ce-97e02d28b294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:30.785365 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-04a0397b-d575-4356-82ce-97e02d28b294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:30.805988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0397b-d575-4356-82ce-97e02d28b294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:30.809789 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0397b-d575-4356-82ce-97e02d28b294 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:30.849558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1152/2342] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:32:33.240642 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bddec-2f4d-4d52-a1e5-128c72908643 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:33.241167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bddec-2f4d-4d52-a1e5-128c72908643 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:33.241271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bddec-2f4d-4d52-a1e5-128c72908643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:33.263768 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-288bddec-2f4d-4d52-a1e5-128c72908643 None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:32:33.264975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-288bddec-2f4d-4d52-a1e5-128c72908643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:33.272530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1191/2343] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:32:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:32:33.285973 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0397b-d575-4356-82ce-97e02d28b294 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:33.300781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0397b-d575-4356-82ce-97e02d28b294 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:33.306392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf2e0c-2f06-4fa4-9eca-33781becd55c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:33.307158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf2e0c-2f06-4fa4-9eca-33781becd55c service swift] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:33.307452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf2e0c-2f06-4fa4-9eca-33781becd55c service swift] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:33.307728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf2e0c-2f06-4fa4-9eca-33781becd55c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:33.318269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf2e0c-2f06-4fa4-9eca-33781becd55c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:33.320068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf2e0c-2f06-4fa4-9eca-33781becd55c service swift] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:33.329695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1153/2344] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:32:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:32:43.257474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=52 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:32:43.326291 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=83 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:32:51.280889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44be2f-f8f8-41e9-9bf6-4b3b8464a8fe None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:51.281793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44be2f-f8f8-41e9-9bf6-4b3b8464a8fe None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:51.281793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44be2f-f8f8-41e9-9bf6-4b3b8464a8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:51.343088 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6c44be2f-f8f8-41e9-9bf6-4b3b8464a8fe None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:32:51.345042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6c44be2f-f8f8-41e9-9bf6-4b3b8464a8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:51.392162 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1192/2345] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:32:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:32:51.419322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf2e0c-2f06-4fa4-9eca-33781becd55c service swift] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:51.492164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf2e0c-2f06-4fa4-9eca-33781becd55c service swift] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:51.504444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85520163-1e73-4d75-a182-eb14563e59ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:51.505413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520163-1e73-4d75-a182-eb14563e59ea service swift] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:51.507045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520163-1e73-4d75-a182-eb14563e59ea service swift] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:51.507045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520163-1e73-4d75-a182-eb14563e59ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:51.523990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85520163-1e73-4d75-a182-eb14563e59ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:51.529065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85520163-1e73-4d75-a182-eb14563e59ea service swift] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:51.547821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1154/2346] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:32:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:32:51.601227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44be2f-f8f8-41e9-9bf6-4b3b8464a8fe None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:32:51.621565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44be2f-f8f8-41e9-9bf6-4b3b8464a8fe None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:32:51.630790 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75e79f-3e29-45a9-99f1-824d94b885b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:32:51.631341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75e79f-3e29-45a9-99f1-824d94b885b0 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:32:51.631603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75e79f-3e29-45a9-99f1-824d94b885b0 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:32:51.632636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75e79f-3e29-45a9-99f1-824d94b885b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:32:51.633984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6c75e79f-3e29-45a9-99f1-824d94b885b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:32:51.649968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75e79f-3e29-45a9-99f1-824d94b885b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:32:51.651999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75e79f-3e29-45a9-99f1-824d94b885b0 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:32:51.667326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1193/2347] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:32:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:33:01.541174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=20 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:33:01.660418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=26 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:33:07.002899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85520163-1e73-4d75-a182-eb14563e59ea service swift] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:07.023077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85520163-1e73-4d75-a182-eb14563e59ea service swift] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:07.032079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a464f17-943d-4437-a404-4cafd9bf2a8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:07.033221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464f17-943d-4437-a404-4cafd9bf2a8a service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:07.034927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464f17-943d-4437-a404-4cafd9bf2a8a service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:07.034927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464f17-943d-4437-a404-4cafd9bf2a8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:07.037206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1a464f17-943d-4437-a404-4cafd9bf2a8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:07.059748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a464f17-943d-4437-a404-4cafd9bf2a8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:07.061876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a464f17-943d-4437-a404-4cafd9bf2a8a service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:07.080475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1155/2348] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:33:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:33:12.411647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead29bb-5aea-421b-80e5-c3615fcf875f None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:12.412242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead29bb-5aea-421b-80e5-c3615fcf875f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:12.412242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead29bb-5aea-421b-80e5-c3615fcf875f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:12.435476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-7ead29bb-5aea-421b-80e5-c3615fcf875f None None] MFA Rules not processed for user `f46cbc7de18c4f9fb6f54e96be53fea0`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:33:12.436827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7ead29bb-5aea-421b-80e5-c3615fcf875f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:12.444416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1194/2349] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:33:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:33:16.982191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5619d-395f-477a-8975-ba2eb9dc0412 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:16.982811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5619d-395f-477a-8975-ba2eb9dc0412 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:16.982924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5619d-395f-477a-8975-ba2eb9dc0412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:17.007915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-acb5619d-395f-477a-8975-ba2eb9dc0412 None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:33:17.009560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-acb5619d-395f-477a-8975-ba2eb9dc0412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:17.019084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1156/2350] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:33:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:33:17.036854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead29bb-5aea-421b-80e5-c3615fcf875f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:17.053330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead29bb-5aea-421b-80e5-c3615fcf875f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:17.062800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52785c56-aba9-48ca-bfbd-98d4a7c6d593 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:17.063535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52785c56-aba9-48ca-bfbd-98d4a7c6d593 service swift] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:17.064032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52785c56-aba9-48ca-bfbd-98d4a7c6d593 service swift] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:17.064370 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52785c56-aba9-48ca-bfbd-98d4a7c6d593 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:17.075763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52785c56-aba9-48ca-bfbd-98d4a7c6d593 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:17.077582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52785c56-aba9-48ca-bfbd-98d4a7c6d593 service swift] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:17.091107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1195/2351] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:33:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:33:19.379567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2f7aa-e5bf-44be-be25-de387fdce2cc None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:19.383503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2f7aa-e5bf-44be-be25-de387fdce2cc None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:19.384092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2f7aa-e5bf-44be-be25-de387fdce2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:19.425726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-00e2f7aa-e5bf-44be-be25-de387fdce2cc None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:33:19.427708 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-00e2f7aa-e5bf-44be-be25-de387fdce2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:19.468008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1157/2352] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 22 17:33:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:33:19.499165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52785c56-aba9-48ca-bfbd-98d4a7c6d593 service swift] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:19.522315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52785c56-aba9-48ca-bfbd-98d4a7c6d593 service swift] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:19.533604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4740272-17e1-4e18-b459-e7f6ddd157aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:19.535218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4740272-17e1-4e18-b459-e7f6ddd157aa service swift] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:19.536056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4740272-17e1-4e18-b459-e7f6ddd157aa service swift] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:19.537049 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4740272-17e1-4e18-b459-e7f6ddd157aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:19.567087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4740272-17e1-4e18-b459-e7f6ddd157aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:19.572359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4740272-17e1-4e18-b459-e7f6ddd157aa service swift] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:19.628619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1196/2353] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:33:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:33:29.412412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=12 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:33:29.586734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=15 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:33:39.949791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb4da6-3368-42f9-a5ea-0717797e07ac None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:39.950438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb4da6-3368-42f9-a5ea-0717797e07ac None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:39.950807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb4da6-3368-42f9-a5ea-0717797e07ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:39.979001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-71fb4da6-3368-42f9-a5ea-0717797e07ac None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:33:39.980356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-71fb4da6-3368-42f9-a5ea-0717797e07ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:39.988354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1158/2354] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 22 17:33:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:33:40.006635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4740272-17e1-4e18-b459-e7f6ddd157aa service swift] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:40.025274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4740272-17e1-4e18-b459-e7f6ddd157aa service swift] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:40.035692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da38992-4f6d-439f-963d-341f587196eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:40.036641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da38992-4f6d-439f-963d-341f587196eb service swift] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:40.037303 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da38992-4f6d-439f-963d-341f587196eb service swift] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:40.037771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da38992-4f6d-439f-963d-341f587196eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:40.051672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da38992-4f6d-439f-963d-341f587196eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:40.057060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da38992-4f6d-439f-963d-341f587196eb service swift] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:40.066096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1197/2355] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:33:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 17:33:41.564168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb4da6-3368-42f9-a5ea-0717797e07ac None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:41.594810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb4da6-3368-42f9-a5ea-0717797e07ac None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:41.600757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf04da-5dfc-4d87-bbc5-cc34e3cd1faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:41.601670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf04da-5dfc-4d87-bbc5-cc34e3cd1faf service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:41.603254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf04da-5dfc-4d87-bbc5-cc34e3cd1faf service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:41.603254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf04da-5dfc-4d87-bbc5-cc34e3cd1faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:41.615577 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf04da-5dfc-4d87-bbc5-cc34e3cd1faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:41.617859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf04da-5dfc-4d87-bbc5-cc34e3cd1faf service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:41.636076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1159/2356] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:33:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:33:43.419403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da38992-4f6d-439f-963d-341f587196eb service swift] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:43.442084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da38992-4f6d-439f-963d-341f587196eb service swift] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:43.453155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6358b-51d4-42bd-a113-6934cdf540ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:43.454410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6358b-51d4-42bd-a113-6934cdf540ef service placement] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:43.455040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6358b-51d4-42bd-a113-6934cdf540ef service placement] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:43.455555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6358b-51d4-42bd-a113-6934cdf540ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:43.459224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fea6358b-51d4-42bd-a113-6934cdf540ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:43.483299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6358b-51d4-42bd-a113-6934cdf540ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:43.485983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6358b-51d4-42bd-a113-6934cdf540ef service placement] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:43.508384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1198/2357] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:33:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:33:43.583519 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf04da-5dfc-4d87-bbc5-cc34e3cd1faf service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:43.586406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1bcf04da-5dfc-4d87-bbc5-cc34e3cd1faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:43.727732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf04da-5dfc-4d87-bbc5-cc34e3cd1faf service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:43.734386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:43.735217 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:43.735592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:43.735953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] PATH_INFO: `/v3/users/2351ea9262c44d74827e79231d1813c6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:43.737434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2351ea9262c44d74827e79231d1813c6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:43.740071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:43.994001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2351ea9262c44d74827e79231d1813c6'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:33:43.995584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2351ea9262c44d74827e79231d1813c6'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:33:43.995873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2351ea9262c44d74827e79231d1813c6'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:33:44.006079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2351ea9262c44d74827e79231d1813c6'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:33:44.028087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-731886bf-d281-4422-9bff-e5b49c83d923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2351ea9262c44d74827e79231d1813c6'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:33:44.143727 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1160/2358] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:33:43 2022] DELETE /identity/v3/users/2351ea9262c44d74827e79231d1813c6 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:33:44.564695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6358b-51d4-42bd-a113-6934cdf540ef service placement] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:44.613756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6358b-51d4-42bd-a113-6934cdf540ef service placement] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:44.766722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:44.768238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:44.769918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:44.770620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] PATH_INFO: `/v3/projects/7508a86facdb4ff98b171aa75962dde5` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:44.773547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7508a86facdb4ff98b171aa75962dde5)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:44.778991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:44.985112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7508a86facdb4ff98b171aa75962dde5', 'request_id': 'req-7f051194-232f-4383-b6bf-9015eedf6a2a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:33:44.996410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7508a86facdb4ff98b171aa75962dde5', 'request_id': 'req-7f051194-232f-4383-b6bf-9015eedf6a2a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:33:45.005752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7508a86facdb4ff98b171aa75962dde5', 'request_id': 'req-7f051194-232f-4383-b6bf-9015eedf6a2a'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:33:45.101681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] The token cache is being invalidate because project 7508a86facdb4ff98b171aa75962dde5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:33:45.102287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:33:45.103568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-7f051194-232f-4383-b6bf-9015eedf6a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:33:45.106812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1199/2359] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:33:44 2022] DELETE /identity/v3/projects/7508a86facdb4ff98b171aa75962dde5 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:33:45.113956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d64c60-ef69-45e3-9b70-0678bc651c60 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:45.114825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d64c60-ef69-45e3-9b70-0678bc651c60 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:45.115077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d64c60-ef69-45e3-9b70-0678bc651c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:45.171358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:33:45.247176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-18d64c60-ef69-45e3-9b70-0678bc651c60 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:33:45.249941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-18d64c60-ef69-45e3-9b70-0678bc651c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:45.362257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1161/2360] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:33:45.433257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24815219-8e79-4265-9392-e503c3a0735a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:45.433721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24815219-8e79-4265-9392-e503c3a0735a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:45.434695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24815219-8e79-4265-9392-e503c3a0735a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:45.455850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-24815219-8e79-4265-9392-e503c3a0735a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:33:45.457227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-24815219-8e79-4265-9392-e503c3a0735a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:45.469692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1200/2361] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:33:45.474056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d64c60-ef69-45e3-9b70-0678bc651c60 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:45.485647 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d64c60-ef69-45e3-9b70-0678bc651c60 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:45.496919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837a4b3-375e-48d4-843c-b4e391e9b603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:45.497913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a4b3-375e-48d4-843c-b4e391e9b603 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:45.502835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a4b3-375e-48d4-843c-b4e391e9b603 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:45.502835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a4b3-375e-48d4-843c-b4e391e9b603 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:45.502835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837a4b3-375e-48d4-843c-b4e391e9b603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:33:45.502835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837a4b3-375e-48d4-843c-b4e391e9b603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:45.502835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837a4b3-375e-48d4-843c-b4e391e9b603 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:45.530885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1162/2362] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:33:45.536018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24815219-8e79-4265-9392-e503c3a0735a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:45.554369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24815219-8e79-4265-9392-e503c3a0735a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:45.563890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a10384-c630-476f-bc22-682f6815ba7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:45.565030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a10384-c630-476f-bc22-682f6815ba7c admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:45.565575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a10384-c630-476f-bc22-682f6815ba7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:45.566008 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a10384-c630-476f-bc22-682f6815ba7c admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:45.567277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a10384-c630-476f-bc22-682f6815ba7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:45.571512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a10384-c630-476f-bc22-682f6815ba7c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:45.812834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1201/2363] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:33:45 2022] POST /identity/v3/projects => generated 373 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:33:45.819063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837a4b3-375e-48d4-843c-b4e391e9b603 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:45.833000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837a4b3-375e-48d4-843c-b4e391e9b603 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:45.935438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067677b3-0ef5-4ae8-a513-0760e2950ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:45.936535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067677b3-0ef5-4ae8-a513-0760e2950ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:45.937101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067677b3-0ef5-4ae8-a513-0760e2950ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:45.937504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067677b3-0ef5-4ae8-a513-0760e2950ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:45.938255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067677b3-0ef5-4ae8-a513-0760e2950ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:45.940660 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067677b3-0ef5-4ae8-a513-0760e2950ee1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:46.090313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1163/2364] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:33:45 2022] POST /identity/v3/users => generated 287 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:33:46.091247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a10384-c630-476f-bc22-682f6815ba7c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:46.108625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a10384-c630-476f-bc22-682f6815ba7c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:46.115186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3763005-883a-4ecb-a176-5068115cb1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:46.115943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3763005-883a-4ecb-a176-5068115cb1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:46.116286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3763005-883a-4ecb-a176-5068115cb1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:46.116599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3763005-883a-4ecb-a176-5068115cb1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:46.117343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3763005-883a-4ecb-a176-5068115cb1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:46.119214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3763005-883a-4ecb-a176-5068115cb1d1 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:46.131707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1202/2365] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:33:46.136294 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067677b3-0ef5-4ae8-a513-0760e2950ee1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:46.147149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067677b3-0ef5-4ae8-a513-0760e2950ee1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:46.152323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55978e9-d8df-4e6e-8af3-63c47af617c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:46.152986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55978e9-d8df-4e6e-8af3-63c47af617c1 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:46.153264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55978e9-d8df-4e6e-8af3-63c47af617c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:46.153530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55978e9-d8df-4e6e-8af3-63c47af617c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:46.154190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55978e9-d8df-4e6e-8af3-63c47af617c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:46.155919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55978e9-d8df-4e6e-8af3-63c47af617c1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:46.177899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1164/2366] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:33:46.182637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3763005-883a-4ecb-a176-5068115cb1d1 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:46.193372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3763005-883a-4ecb-a176-5068115cb1d1 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:46.199015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a5356-db4b-4706-870e-5620ef07b8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:46.199698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a5356-db4b-4706-870e-5620ef07b8ef admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:46.199980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a5356-db4b-4706-870e-5620ef07b8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:46.200260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a5356-db4b-4706-870e-5620ef07b8ef admin admin] PATH_INFO: `/v3/projects/956e52f3b5824c6b8d473ae0fef859c4/users/59d328d707b7409288cd118aecd6af54/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:46.215818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:33:46.219227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a5356-db4b-4706-870e-5620ef07b8ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=956e52f3b5824c6b8d473ae0fef859c4, user_id=59d328d707b7409288cd118aecd6af54, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:46.222053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a5356-db4b-4706-870e-5620ef07b8ef admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:46.335306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1203/2367] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:33:46 2022] PUT /identity/v3/projects/956e52f3b5824c6b8d473ae0fef859c4/users/59d328d707b7409288cd118aecd6af54/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:33:46.346144 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8261e-a4f8-4560-8738-02981e2895f1 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:46.346431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8261e-a4f8-4560-8738-02981e2895f1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:46.346734 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8261e-a4f8-4560-8738-02981e2895f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:46.378796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-29e8261e-a4f8-4560-8738-02981e2895f1 None None] MFA Rules not processed for user `59d328d707b7409288cd118aecd6af54`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:33:46.380080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-29e8261e-a4f8-4560-8738-02981e2895f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:46.434587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:33:46.462402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1165/2368] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:33:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:33:46.479512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a5356-db4b-4706-870e-5620ef07b8ef admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:33:46.481603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ac6a5356-db4b-4706-870e-5620ef07b8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:46.602801 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a5356-db4b-4706-870e-5620ef07b8ef admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:33:46.609854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e487bea-3562-405f-8636-b651fd31fbe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:33:46.610903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e487bea-3562-405f-8636-b651fd31fbe1 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:46.611375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e487bea-3562-405f-8636-b651fd31fbe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:46.611857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e487bea-3562-405f-8636-b651fd31fbe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:46.627776 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e487bea-3562-405f-8636-b651fd31fbe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:33:46.630214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e487bea-3562-405f-8636-b651fd31fbe1 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:33:46.648552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1204/2369] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:33:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:33:48.536914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6166c0d-974e-4ab7-8a39-8e620cbae0b8 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:33:48.537573 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6166c0d-974e-4ab7-8a39-8e620cbae0b8 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:33:48.537573 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6166c0d-974e-4ab7-8a39-8e620cbae0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:33:48.559590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b6166c0d-974e-4ab7-8a39-8e620cbae0b8 None None] MFA Rules not processed for user `59d328d707b7409288cd118aecd6af54`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:33:48.560929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b6166c0d-974e-4ab7-8a39-8e620cbae0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:33:48.568371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1166/2370] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:33:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:33:56.640382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:33:58.553613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=9 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:34:10.617072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d01c5c-440f-477d-a276-6d6bd87ed1f9 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:10.618105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d01c5c-440f-477d-a276-6d6bd87ed1f9 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:10.618105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d01c5c-440f-477d-a276-6d6bd87ed1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:10.646975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-56d01c5c-440f-477d-a276-6d6bd87ed1f9 None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:10.648427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-56d01c5c-440f-477d-a276-6d6bd87ed1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:10.738176 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1205/2371] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:34:10.763189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6166c0d-974e-4ab7-8a39-8e620cbae0b8 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:10.765087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b6166c0d-974e-4ab7-8a39-8e620cbae0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:10.864749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6166c0d-974e-4ab7-8a39-8e620cbae0b8 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:10.870218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958b5a0-c45d-4c63-94fd-e493d238f491 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:10.870889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958b5a0-c45d-4c63-94fd-e493d238f491 service swift] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:10.871167 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958b5a0-c45d-4c63-94fd-e493d238f491 service swift] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:10.871473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958b5a0-c45d-4c63-94fd-e493d238f491 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:10.880964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958b5a0-c45d-4c63-94fd-e493d238f491 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:10.882827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958b5a0-c45d-4c63-94fd-e493d238f491 service swift] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:10.893259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1167/2372] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:34:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:34:12.457026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f276a-7c79-4f5b-a640-cce5c55cbf23 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:12.458323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f276a-7c79-4f5b-a640-cce5c55cbf23 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:12.458861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f276a-7c79-4f5b-a640-cce5c55cbf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:12.492174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0a0f276a-7c79-4f5b-a640-cce5c55cbf23 None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:12.494241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0a0f276a-7c79-4f5b-a640-cce5c55cbf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:12.506938 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1206/2373] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 22 17:34:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:34:12.526833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958b5a0-c45d-4c63-94fd-e493d238f491 service swift] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:12.546429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958b5a0-c45d-4c63-94fd-e493d238f491 service swift] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:12.553820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6d02c-505d-404d-aae4-490207cb7164 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:12.554525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6d02c-505d-404d-aae4-490207cb7164 service swift] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:12.554919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6d02c-505d-404d-aae4-490207cb7164 service swift] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:12.555288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6d02c-505d-404d-aae4-490207cb7164 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:12.569327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6d02c-505d-404d-aae4-490207cb7164 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:12.573292 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6d02c-505d-404d-aae4-490207cb7164 service swift] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:12.590746 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1168/2374] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:34:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:34:13.414990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dcd0c-be96-4690-b159-d3c8060b0c9d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:13.416276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dcd0c-be96-4690-b159-d3c8060b0c9d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:13.416276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dcd0c-be96-4690-b159-d3c8060b0c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:13.440375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8e5dcd0c-be96-4690-b159-d3c8060b0c9d None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:13.442235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8e5dcd0c-be96-4690-b159-d3c8060b0c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:13.454616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1207/2375] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:34:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:34:13.471007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6d02c-505d-404d-aae4-490207cb7164 service swift] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:13.489249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6d02c-505d-404d-aae4-490207cb7164 service swift] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:13.498386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45595e9c-8514-4fbe-933a-1113b4479c2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:13.499951 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45595e9c-8514-4fbe-933a-1113b4479c2d service swift] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:13.500437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45595e9c-8514-4fbe-933a-1113b4479c2d service swift] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:13.500906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45595e9c-8514-4fbe-933a-1113b4479c2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:13.515909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45595e9c-8514-4fbe-933a-1113b4479c2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:13.518738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45595e9c-8514-4fbe-933a-1113b4479c2d service swift] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:13.531040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1169/2376] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:34:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 17:34:14.399186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d64a0-5447-4667-8cd1-88bcc7d96557 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:14.400081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d64a0-5447-4667-8cd1-88bcc7d96557 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:14.400273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d64a0-5447-4667-8cd1-88bcc7d96557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:14.428243 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8a4d64a0-5447-4667-8cd1-88bcc7d96557 None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:14.430229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8a4d64a0-5447-4667-8cd1-88bcc7d96557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:14.441478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1208/2377] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 22 17:34:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:34:14.464624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45595e9c-8514-4fbe-933a-1113b4479c2d service swift] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:14.480157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45595e9c-8514-4fbe-933a-1113b4479c2d service swift] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:14.486266 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2eafb-ef31-42d4-b437-040ce6dd038a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:14.486938 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2eafb-ef31-42d4-b437-040ce6dd038a service swift] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:14.487210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2eafb-ef31-42d4-b437-040ce6dd038a service swift] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:14.487469 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2eafb-ef31-42d4-b437-040ce6dd038a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:14.499141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2eafb-ef31-42d4-b437-040ce6dd038a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:14.501081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2eafb-ef31-42d4-b437-040ce6dd038a service swift] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:14.512317 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1170/2378] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:34:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:34:17.808832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d64a0-5447-4667-8cd1-88bcc7d96557 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:17.903148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d64a0-5447-4667-8cd1-88bcc7d96557 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:17.908972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:17.909762 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:17.910099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:17.910606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] PATH_INFO: `/v3/users/59d328d707b7409288cd118aecd6af54` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:17.912416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d328d707b7409288cd118aecd6af54)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:17.914848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:18.195608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d328d707b7409288cd118aecd6af54'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:34:18.196504 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d328d707b7409288cd118aecd6af54'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:34:18.197257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d328d707b7409288cd118aecd6af54'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:34:18.354208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d328d707b7409288cd118aecd6af54'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:34:18.365539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d328d707b7409288cd118aecd6af54'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:34:18.388417 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1209/2379] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:34:17 2022] DELETE /identity/v3/users/59d328d707b7409288cd118aecd6af54 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:34:18.393278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4be1ce-89f8-473f-b52b-ffd48fce716c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:18.393887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4be1ce-89f8-473f-b52b-ffd48fce716c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:18.393887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4be1ce-89f8-473f-b52b-ffd48fce716c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:18.414797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-bf4be1ce-89f8-473f-b52b-ffd48fce716c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:18.416997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bf4be1ce-89f8-473f-b52b-ffd48fce716c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:18.503308 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1171/2380] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:34:18.515198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:18.517321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:18.523790 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:34:18.603854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbe910-eaee-4abc-8b78-0d1725b7c517 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:18.607786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7484bb-ec10-4027-ac68-3ecaa5ee3a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:18.608422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7484bb-ec10-4027-ac68-3ecaa5ee3a98 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:18.608693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7484bb-ec10-4027-ac68-3ecaa5ee3a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:18.608960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7484bb-ec10-4027-ac68-3ecaa5ee3a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:18.621854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7484bb-ec10-4027-ac68-3ecaa5ee3a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:18.623418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7484bb-ec10-4027-ac68-3ecaa5ee3a98 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:18.638864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1210/2381] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:34:18.945575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4be1ce-89f8-473f-b52b-ffd48fce716c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:18.947920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bf4be1ce-89f8-473f-b52b-ffd48fce716c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:18.969526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4be1ce-89f8-473f-b52b-ffd48fce716c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:18.977801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:18.979250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:18.979628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:18.980118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] PATH_INFO: `/v3/projects/956e52f3b5824c6b8d473ae0fef859c4` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:18.982311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=956e52f3b5824c6b8d473ae0fef859c4)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:18.986185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:19.023059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da65380-a5df-4f52-afad-75cbb5d7be06 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:19.023889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da65380-a5df-4f52-afad-75cbb5d7be06 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:19.023889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da65380-a5df-4f52-afad-75cbb5d7be06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:19.046600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0da65380-a5df-4f52-afad-75cbb5d7be06 None None] MFA Rules not processed for user `b0db362075b04b2294a6268576a5bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:19.047833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0da65380-a5df-4f52-afad-75cbb5d7be06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:19.119305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:34:19.120757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1211/2382] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:34:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:34:19.123733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '956e52f3b5824c6b8d473ae0fef859c4', 'request_id': 'req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:34:19.132061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '956e52f3b5824c6b8d473ae0fef859c4', 'request_id': 'req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:34:19.295091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '956e52f3b5824c6b8d473ae0fef859c4', 'request_id': 'req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:34:19.312970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] The token cache is being invalidate because project 956e52f3b5824c6b8d473ae0fef859c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:34:19.313578 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:34:19.314772 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-eda42a3f-f1bb-475e-909e-0560b8c5e2bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:34:19.317415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1172/2383] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:34:18 2022] DELETE /identity/v3/projects/956e52f3b5824c6b8d473ae0fef859c4 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:34:19.324191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166f1c7-d74f-4992-968f-411fe05feed5 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:19.324692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166f1c7-d74f-4992-968f-411fe05feed5 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:19.325185 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166f1c7-d74f-4992-968f-411fe05feed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:19.364854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d166f1c7-d74f-4992-968f-411fe05feed5 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:19.366306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d166f1c7-d74f-4992-968f-411fe05feed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:19.450360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1212/2384] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:34:19.458736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d618635-db17-40a4-be7f-485f2ac0dcbb None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:19.459505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d618635-db17-40a4-be7f-485f2ac0dcbb None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:19.459960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d618635-db17-40a4-be7f-485f2ac0dcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:19.483172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-0d618635-db17-40a4-be7f-485f2ac0dcbb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:19.484447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0d618635-db17-40a4-be7f-485f2ac0dcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:19.496923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1173/2385] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:34:19.502689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166f1c7-d74f-4992-968f-411fe05feed5 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:19.515020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166f1c7-d74f-4992-968f-411fe05feed5 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:19.520215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292bcc8-bd5b-487d-89e0-6c4d6aceddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:19.520869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292bcc8-bd5b-487d-89e0-6c4d6aceddfb admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:19.521145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292bcc8-bd5b-487d-89e0-6c4d6aceddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:19.521421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292bcc8-bd5b-487d-89e0-6c4d6aceddfb admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:19.522102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292bcc8-bd5b-487d-89e0-6c4d6aceddfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:34:19.522400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292bcc8-bd5b-487d-89e0-6c4d6aceddfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:19.524141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292bcc8-bd5b-487d-89e0-6c4d6aceddfb admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:19.542063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1213/2386] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:34:19.547200 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d618635-db17-40a4-be7f-485f2ac0dcbb None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:19.558103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d618635-db17-40a4-be7f-485f2ac0dcbb None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:19.563205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad611e4-ae62-4bf4-b41c-a271a9b11627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:19.563824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad611e4-ae62-4bf4-b41c-a271a9b11627 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:19.564104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad611e4-ae62-4bf4-b41c-a271a9b11627 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:19.564375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad611e4-ae62-4bf4-b41c-a271a9b11627 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:19.565017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad611e4-ae62-4bf4-b41c-a271a9b11627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:19.567055 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad611e4-ae62-4bf4-b41c-a271a9b11627 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:19.673079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1174/2387] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:34:19 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:34:19.677920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292bcc8-bd5b-487d-89e0-6c4d6aceddfb admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:19.689923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292bcc8-bd5b-487d-89e0-6c4d6aceddfb admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:19.726328 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:34:19.773184 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fa61d-113b-414e-b482-309c3cdcf6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:19.773983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fa61d-113b-414e-b482-309c3cdcf6fa admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:19.775854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fa61d-113b-414e-b482-309c3cdcf6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:19.775854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fa61d-113b-414e-b482-309c3cdcf6fa admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:19.776191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fa61d-113b-414e-b482-309c3cdcf6fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:19.778352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fa61d-113b-414e-b482-309c3cdcf6fa admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:19.948753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1214/2388] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:34:19 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:34:19.953271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad611e4-ae62-4bf4-b41c-a271a9b11627 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:19.965925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad611e4-ae62-4bf4-b41c-a271a9b11627 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:19.971035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb4b2a-7d27-49c0-bc07-b2f7df3ab551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:19.971681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb4b2a-7d27-49c0-bc07-b2f7df3ab551 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:19.972007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb4b2a-7d27-49c0-bc07-b2f7df3ab551 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:19.972272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb4b2a-7d27-49c0-bc07-b2f7df3ab551 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:19.972949 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb4b2a-7d27-49c0-bc07-b2f7df3ab551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:19.974692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb4b2a-7d27-49c0-bc07-b2f7df3ab551 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:19.987358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1175/2389] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:34:19.993045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fa61d-113b-414e-b482-309c3cdcf6fa admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:20.004511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fa61d-113b-414e-b482-309c3cdcf6fa admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:20.011406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112af976-8f97-41c1-91ed-7d9e7c2e8bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:20.012149 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112af976-8f97-41c1-91ed-7d9e7c2e8bde admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:20.012428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112af976-8f97-41c1-91ed-7d9e7c2e8bde admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:20.012786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112af976-8f97-41c1-91ed-7d9e7c2e8bde admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:20.013380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112af976-8f97-41c1-91ed-7d9e7c2e8bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:20.015220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112af976-8f97-41c1-91ed-7d9e7c2e8bde admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:20.029087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1215/2390] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:34:20.035059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb4b2a-7d27-49c0-bc07-b2f7df3ab551 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:20.049203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb4b2a-7d27-49c0-bc07-b2f7df3ab551 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:20.057341 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a0aef-85f3-45a0-812c-fea416628564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:20.058521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a0aef-85f3-45a0-812c-fea416628564 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:20.059546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a0aef-85f3-45a0-812c-fea416628564 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:20.059546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a0aef-85f3-45a0-812c-fea416628564 admin admin] PATH_INFO: `/v3/projects/43d4499a9d9f4c00a09c867bb403dd30/users/aa55b9fade0047c68c2f3f6bf4dfe44e/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:20.088523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a0aef-85f3-45a0-812c-fea416628564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d4499a9d9f4c00a09c867bb403dd30, user_id=aa55b9fade0047c68c2f3f6bf4dfe44e, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:20.093580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a0aef-85f3-45a0-812c-fea416628564 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:20.194137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1176/2391] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:34:20 2022] PUT /identity/v3/projects/43d4499a9d9f4c00a09c867bb403dd30/users/aa55b9fade0047c68c2f3f6bf4dfe44e/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:34:20.202468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e330d-3c7f-48d2-b246-589e245ef714 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:20.203197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e330d-3c7f-48d2-b246-589e245ef714 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:20.203589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e330d-3c7f-48d2-b246-589e245ef714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:20.240274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-4e3e330d-3c7f-48d2-b246-589e245ef714 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:20.242418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4e3e330d-3c7f-48d2-b246-589e245ef714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:20.347470 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1216/2392] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:34:20.360459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a0aef-85f3-45a0-812c-fea416628564 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:20.363422 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e51a0aef-85f3-45a0-812c-fea416628564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:20.455187 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a0aef-85f3-45a0-812c-fea416628564 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:20.459055 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad862f-baf6-42b6-81dd-ac90f2a53f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:20.460082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad862f-baf6-42b6-81dd-ac90f2a53f71 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:20.460082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad862f-baf6-42b6-81dd-ac90f2a53f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:20.460417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad862f-baf6-42b6-81dd-ac90f2a53f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:20.471424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad862f-baf6-42b6-81dd-ac90f2a53f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:20.472893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad862f-baf6-42b6-81dd-ac90f2a53f71 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:20.485819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1177/2393] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:34:29.569175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c78d9-dafd-4bad-a5fe-7ec631c24787 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:29.569773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c78d9-dafd-4bad-a5fe-7ec631c24787 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:29.569773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c78d9-dafd-4bad-a5fe-7ec631c24787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:29.610691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-188c78d9-dafd-4bad-a5fe-7ec631c24787 None None] MFA Rules not processed for user `aa55b9fade0047c68c2f3f6bf4dfe44e`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:29.612050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-188c78d9-dafd-4bad-a5fe-7ec631c24787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:29.708066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1217/2394] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:34:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:34:29.714361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad862f-baf6-42b6-81dd-ac90f2a53f71 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:29.728520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad862f-baf6-42b6-81dd-ac90f2a53f71 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:29.735575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddd962-86ad-444c-a1ea-5fb0823aa7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:29.736344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddd962-86ad-444c-a1ea-5fb0823aa7eb admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:29.736742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddd962-86ad-444c-a1ea-5fb0823aa7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:29.737261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddd962-86ad-444c-a1ea-5fb0823aa7eb admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:29.738014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddd962-86ad-444c-a1ea-5fb0823aa7eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:29.741342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddd962-86ad-444c-a1ea-5fb0823aa7eb admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:29.949432 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c78d9-dafd-4bad-a5fe-7ec631c24787 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:29.958589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1178/2395] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:34:29 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:34:29.971073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c78d9-dafd-4bad-a5fe-7ec631c24787 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:30.036206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:34:30.085262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18530cc5-24cc-47e1-b3fe-a1c856037e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:30.087877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18530cc5-24cc-47e1-b3fe-a1c856037e19 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:30.088459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18530cc5-24cc-47e1-b3fe-a1c856037e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:30.088875 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18530cc5-24cc-47e1-b3fe-a1c856037e19 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:30.089845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18530cc5-24cc-47e1-b3fe-a1c856037e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:30.093622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18530cc5-24cc-47e1-b3fe-a1c856037e19 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:30.210450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1218/2396] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:34:29 2022] POST /identity/v3/users => generated 295 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:34:30.216826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddd962-86ad-444c-a1ea-5fb0823aa7eb admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:30.236476 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddd962-86ad-444c-a1ea-5fb0823aa7eb admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:30.245297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73252ac0-afec-494f-bc55-4d500b4b5c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:30.246555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73252ac0-afec-494f-bc55-4d500b4b5c29 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:30.247102 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73252ac0-afec-494f-bc55-4d500b4b5c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:30.247582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73252ac0-afec-494f-bc55-4d500b4b5c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:30.248924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73252ac0-afec-494f-bc55-4d500b4b5c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:30.252089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73252ac0-afec-494f-bc55-4d500b4b5c29 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:30.271982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1179/2397] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:34:30.278097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18530cc5-24cc-47e1-b3fe-a1c856037e19 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:30.298095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18530cc5-24cc-47e1-b3fe-a1c856037e19 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:30.306747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc83ae6-1bdc-405d-a489-07845305750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:30.308068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc83ae6-1bdc-405d-a489-07845305750f admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:30.308374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc83ae6-1bdc-405d-a489-07845305750f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:30.308844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc83ae6-1bdc-405d-a489-07845305750f admin admin] PATH_INFO: `/v3/projects/51cb74740c3a44159fc3536014a8dfc8/users/d153cab8088641b896e6d4e92b57848c/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:30.353540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc83ae6-1bdc-405d-a489-07845305750f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51cb74740c3a44159fc3536014a8dfc8, user_id=d153cab8088641b896e6d4e92b57848c, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:30.367102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc83ae6-1bdc-405d-a489-07845305750f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:30.536626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1219/2398] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:34:30 2022] PUT /identity/v3/projects/51cb74740c3a44159fc3536014a8dfc8/users/d153cab8088641b896e6d4e92b57848c/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:34:30.543558 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73252ac0-afec-494f-bc55-4d500b4b5c29 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:30.568443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73252ac0-afec-494f-bc55-4d500b4b5c29 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:30.693072 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:34:30.739414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b183c75-882a-4c0c-a691-65057c4f4769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:30.741956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b183c75-882a-4c0c-a691-65057c4f4769 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:30.743224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b183c75-882a-4c0c-a691-65057c4f4769 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:30.747902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b183c75-882a-4c0c-a691-65057c4f4769 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:30.747902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b183c75-882a-4c0c-a691-65057c4f4769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:30.749329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b183c75-882a-4c0c-a691-65057c4f4769 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:30.771934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1180/2399] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:34:30.778918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc83ae6-1bdc-405d-a489-07845305750f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:30.800216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc83ae6-1bdc-405d-a489-07845305750f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:30.813632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37f3e3-9b2a-4fc9-98d6-04847b1ba77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:30.814926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37f3e3-9b2a-4fc9-98d6-04847b1ba77f admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:30.816003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37f3e3-9b2a-4fc9-98d6-04847b1ba77f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:30.816003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37f3e3-9b2a-4fc9-98d6-04847b1ba77f admin admin] PATH_INFO: `/v3/projects/51cb74740c3a44159fc3536014a8dfc8/users/d153cab8088641b896e6d4e92b57848c/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:30.819954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37f3e3-9b2a-4fc9-98d6-04847b1ba77f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51cb74740c3a44159fc3536014a8dfc8, user_id=d153cab8088641b896e6d4e92b57848c, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:30.825459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37f3e3-9b2a-4fc9-98d6-04847b1ba77f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:30.854779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1220/2400] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:34:30 2022] PUT /identity/v3/projects/51cb74740c3a44159fc3536014a8dfc8/users/d153cab8088641b896e6d4e92b57848c/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:34:31.946019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b183c75-882a-4c0c-a691-65057c4f4769 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:31.949451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1b183c75-882a-4c0c-a691-65057c4f4769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:32.138971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b183c75-882a-4c0c-a691-65057c4f4769 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:32.259446 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aca278-e69e-4274-bf0e-d7a6195fcf7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:32.261263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca278-e69e-4274-bf0e-d7a6195fcf7f service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:32.261263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca278-e69e-4274-bf0e-d7a6195fcf7f service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:32.261817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca278-e69e-4274-bf0e-d7a6195fcf7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:32.265451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-95aca278-e69e-4274-bf0e-d7a6195fcf7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:32.294203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aca278-e69e-4274-bf0e-d7a6195fcf7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:32.297612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aca278-e69e-4274-bf0e-d7a6195fcf7f service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:32.322588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1181/2401] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:34:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:34:40.845052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=27,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:34:41.031098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388587f-b33b-4146-b1ec-288c86717e81 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:41.031098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388587f-b33b-4146-b1ec-288c86717e81 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:41.031098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388587f-b33b-4146-b1ec-288c86717e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:41.075772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e388587f-b33b-4146-b1ec-288c86717e81 None None] MFA Rules not processed for user `d153cab8088641b896e6d4e92b57848c`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:41.077744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e388587f-b33b-4146-b1ec-288c86717e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:41.231816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1221/2402] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:34:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:34:41.239712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aca278-e69e-4274-bf0e-d7a6195fcf7f service placement] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:34:41.246174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-95aca278-e69e-4274-bf0e-d7a6195fcf7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:41.416527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:34:41.561042 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aca278-e69e-4274-bf0e-d7a6195fcf7f service placement] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:34:41.573318 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e81b0-439f-4333-8d33-5b63c5ba1ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:34:41.575944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e81b0-439f-4333-8d33-5b63c5ba1ded service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:41.577020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e81b0-439f-4333-8d33-5b63c5ba1ded service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:41.577788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e81b0-439f-4333-8d33-5b63c5ba1ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:41.766533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e81b0-439f-4333-8d33-5b63c5ba1ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:34:41.770064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e81b0-439f-4333-8d33-5b63c5ba1ded service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:34:41.783527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1182/2403] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:34:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:34:48.728160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b048027-bb27-404f-98ac-db853d6794d8 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:34:48.728769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b048027-bb27-404f-98ac-db853d6794d8 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:34:48.728894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b048027-bb27-404f-98ac-db853d6794d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:34:48.777206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-5b048027-bb27-404f-98ac-db853d6794d8 None None] MFA Rules not processed for user `864d7f47381e4e4498ad110dbfcfc154`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:34:48.779482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5b048027-bb27-404f-98ac-db853d6794d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:34:48.871939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1222/2404] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:34:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:34:51.777790 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=46 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:34:58.864007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=59 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:08.907719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e81b0-439f-4333-8d33-5b63c5ba1ded service nova] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:08.909394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c24e81b0-439f-4333-8d33-5b63c5ba1ded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:09.008713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e81b0-439f-4333-8d33-5b63c5ba1ded service nova] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:09.012676 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a0709-8266-4b55-b462-7bc9ad81e40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:09.013306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a0709-8266-4b55-b462-7bc9ad81e40e service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:09.013579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a0709-8266-4b55-b462-7bc9ad81e40e service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:09.013850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a0709-8266-4b55-b462-7bc9ad81e40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:09.016375 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1d8a0709-8266-4b55-b462-7bc9ad81e40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:09.137536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a0709-8266-4b55-b462-7bc9ad81e40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:09.140057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a0709-8266-4b55-b462-7bc9ad81e40e service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:09.160048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1183/2405] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:35:11.693164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fcea5-3285-4925-9acf-9094df146d10 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:11.693918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fcea5-3285-4925-9acf-9094df146d10 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:11.693918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fcea5-3285-4925-9acf-9094df146d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:11.716726 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-551fcea5-3285-4925-9acf-9094df146d10 None None] MFA Rules not processed for user `aa55b9fade0047c68c2f3f6bf4dfe44e`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:11.717952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-551fcea5-3285-4925-9acf-9094df146d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:11.726108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1223/2406] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:35:15.421538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a0709-8266-4b55-b462-7bc9ad81e40e service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:15.436450 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a0709-8266-4b55-b462-7bc9ad81e40e service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:15.442788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647b24b-c339-459f-9142-35396c685668 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:15.444781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647b24b-c339-459f-9142-35396c685668 service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:15.444781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647b24b-c339-459f-9142-35396c685668 service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:15.444781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647b24b-c339-459f-9142-35396c685668 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:15.446812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1647b24b-c339-459f-9142-35396c685668 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:15.463820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647b24b-c339-459f-9142-35396c685668 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:15.465493 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647b24b-c339-459f-9142-35396c685668 service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:15.477741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1184/2407] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:35:15.565094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fcea5-3285-4925-9acf-9094df146d10 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:15.567865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-551fcea5-3285-4925-9acf-9094df146d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:15.598528 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fcea5-3285-4925-9acf-9094df146d10 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:15.608367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:15.609922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:15.610763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:15.611436 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] PATH_INFO: `/v3/users/f46cbc7de18c4f9fb6f54e96be53fea0` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:15.613895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f46cbc7de18c4f9fb6f54e96be53fea0)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:15.620784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:15.965207 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f46cbc7de18c4f9fb6f54e96be53fea0'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:15.965984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f46cbc7de18c4f9fb6f54e96be53fea0'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:15.966696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f46cbc7de18c4f9fb6f54e96be53fea0'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:16.070110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f46cbc7de18c4f9fb6f54e96be53fea0'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:16.080939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dd5a32ef-718d-4eb2-80fa-0961892aa1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f46cbc7de18c4f9fb6f54e96be53fea0'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:16.102604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1224/2408] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:35:15 2022] DELETE /identity/v3/users/f46cbc7de18c4f9fb6f54e96be53fea0 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:16.381230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647b24b-c339-459f-9142-35396c685668 service placement] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:16.402511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647b24b-c339-459f-9142-35396c685668 service placement] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:16.496545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:16.497392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:16.497392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:16.497698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] PATH_INFO: `/v3/projects/de12c2563d50474cac799d50041a2fa0` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:16.498859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de12c2563d50474cac799d50041a2fa0)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:16.500843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:16.637858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:16.651377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de12c2563d50474cac799d50041a2fa0', 'request_id': 'req-acf3a9df-a566-4d09-a507-4b7c60b70fb4'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:16.659503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de12c2563d50474cac799d50041a2fa0', 'request_id': 'req-acf3a9df-a566-4d09-a507-4b7c60b70fb4'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:16.759989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de12c2563d50474cac799d50041a2fa0', 'request_id': 'req-acf3a9df-a566-4d09-a507-4b7c60b70fb4'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:16.780956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] The token cache is being invalidate because project de12c2563d50474cac799d50041a2fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:35:16.781419 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:16.782462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-acf3a9df-a566-4d09-a507-4b7c60b70fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:16.785035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1185/2409] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:35:16 2022] DELETE /identity/v3/projects/de12c2563d50474cac799d50041a2fa0 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:16.799847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453684c2-8aa7-4ef4-b608-01ef1828ebbc None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:16.802124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453684c2-8aa7-4ef4-b608-01ef1828ebbc None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:16.802124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453684c2-8aa7-4ef4-b608-01ef1828ebbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:16.824747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-453684c2-8aa7-4ef4-b608-01ef1828ebbc None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:16.826410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-453684c2-8aa7-4ef4-b608-01ef1828ebbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:16.933528 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1225/2410] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:35:16.936313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0a50c-a127-4875-bf12-1f5d94f1c047 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:16.936788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0a50c-a127-4875-bf12-1f5d94f1c047 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:16.937269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0a50c-a127-4875-bf12-1f5d94f1c047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:16.961665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2cb0a50c-a127-4875-bf12-1f5d94f1c047 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:16.962971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2cb0a50c-a127-4875-bf12-1f5d94f1c047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:16.980260 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1186/2411] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:16.983729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453684c2-8aa7-4ef4-b608-01ef1828ebbc None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:16.998738 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453684c2-8aa7-4ef4-b608-01ef1828ebbc None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:17.005450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b68f43-ed26-455f-9e1d-b47293ca2434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:17.008487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b68f43-ed26-455f-9e1d-b47293ca2434 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:17.008487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b68f43-ed26-455f-9e1d-b47293ca2434 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:17.008487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b68f43-ed26-455f-9e1d-b47293ca2434 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:17.009163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b68f43-ed26-455f-9e1d-b47293ca2434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:35:17.009689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b68f43-ed26-455f-9e1d-b47293ca2434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:17.015562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b68f43-ed26-455f-9e1d-b47293ca2434 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:17.033332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1226/2412] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:35:17.039972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0a50c-a127-4875-bf12-1f5d94f1c047 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:17.054031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0a50c-a127-4875-bf12-1f5d94f1c047 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:17.061563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bc930-5303-4c00-8311-0bf523e6ac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:17.062623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bc930-5303-4c00-8311-0bf523e6ac75 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:17.063101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bc930-5303-4c00-8311-0bf523e6ac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:17.065986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bc930-5303-4c00-8311-0bf523e6ac75 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:17.065986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bc930-5303-4c00-8311-0bf523e6ac75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:17.068517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bc930-5303-4c00-8311-0bf523e6ac75 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:17.196645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1187/2413] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:35:17 2022] POST /identity/v3/projects => generated 403 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:17.202706 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b68f43-ed26-455f-9e1d-b47293ca2434 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:17.219737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b68f43-ed26-455f-9e1d-b47293ca2434 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:17.284814 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:17.298121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7890bd-1b44-43de-b915-9e61e61644d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:17.299047 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7890bd-1b44-43de-b915-9e61e61644d1 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:17.299541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7890bd-1b44-43de-b915-9e61e61644d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:17.299955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7890bd-1b44-43de-b915-9e61e61644d1 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:17.300676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7890bd-1b44-43de-b915-9e61e61644d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:17.303454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7890bd-1b44-43de-b915-9e61e61644d1 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:17.403272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1227/2414] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:35:17 2022] POST /identity/v3/users => generated 302 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:17.407776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bc930-5303-4c00-8311-0bf523e6ac75 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:17.420506 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bc930-5303-4c00-8311-0bf523e6ac75 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:17.426955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90737181-e39d-4f81-9034-eb41604c42f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:17.427636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737181-e39d-4f81-9034-eb41604c42f6 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:17.434629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737181-e39d-4f81-9034-eb41604c42f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:17.434629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737181-e39d-4f81-9034-eb41604c42f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:17.434629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90737181-e39d-4f81-9034-eb41604c42f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:17.434629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90737181-e39d-4f81-9034-eb41604c42f6 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:17.452196 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1188/2415] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:35:17.453500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7890bd-1b44-43de-b915-9e61e61644d1 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:17.467287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7890bd-1b44-43de-b915-9e61e61644d1 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:17.474906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcadc59-ae22-448a-814f-469085ec406a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:17.476507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcadc59-ae22-448a-814f-469085ec406a admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:17.476902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcadc59-ae22-448a-814f-469085ec406a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:17.477381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcadc59-ae22-448a-814f-469085ec406a admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:17.478280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcadc59-ae22-448a-814f-469085ec406a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:17.480410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcadc59-ae22-448a-814f-469085ec406a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:17.499375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1228/2416] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:35:17.501170 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90737181-e39d-4f81-9034-eb41604c42f6 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:17.512678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90737181-e39d-4f81-9034-eb41604c42f6 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:17.517637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cbe0f-6fa7-4c95-be27-86f52db460b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:17.518350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cbe0f-6fa7-4c95-be27-86f52db460b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:17.518971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cbe0f-6fa7-4c95-be27-86f52db460b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:17.519206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cbe0f-6fa7-4c95-be27-86f52db460b7 admin admin] PATH_INFO: `/v3/projects/22862455fed848cb85fff5e74121b1fe/users/e45da558ca4c4deb8755224624cfbec2/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:17.536546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cbe0f-6fa7-4c95-be27-86f52db460b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22862455fed848cb85fff5e74121b1fe, user_id=e45da558ca4c4deb8755224624cfbec2, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:17.539905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cbe0f-6fa7-4c95-be27-86f52db460b7 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:17.653543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1189/2417] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:17 2022] PUT /identity/v3/projects/22862455fed848cb85fff5e74121b1fe/users/e45da558ca4c4deb8755224624cfbec2/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:17.676755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e06e4-6024-4d7f-90e9-786fb5defcfd None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:17.676755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e06e4-6024-4d7f-90e9-786fb5defcfd None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:17.677200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e06e4-6024-4d7f-90e9-786fb5defcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:17.735268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-9d4e06e4-6024-4d7f-90e9-786fb5defcfd None None] MFA Rules not processed for user `e45da558ca4c4deb8755224624cfbec2`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:17.737683 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9d4e06e4-6024-4d7f-90e9-786fb5defcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:17.878794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1229/2418] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:17.884753 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cbe0f-6fa7-4c95-be27-86f52db460b7 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:18.042202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cbe0f-6fa7-4c95-be27-86f52db460b7 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:18.050557 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26b75d-d089-479c-959f-8d5e0e0bd409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:18.051747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26b75d-d089-479c-959f-8d5e0e0bd409 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:18.052307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26b75d-d089-479c-959f-8d5e0e0bd409 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:18.052911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26b75d-d089-479c-959f-8d5e0e0bd409 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:18.054275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26b75d-d089-479c-959f-8d5e0e0bd409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:18.058357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26b75d-d089-479c-959f-8d5e0e0bd409 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:18.228582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1190/2419] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:35:17 2022] POST /identity/v3/projects => generated 405 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:18.233620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e06e4-6024-4d7f-90e9-786fb5defcfd None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:18.258940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e06e4-6024-4d7f-90e9-786fb5defcfd None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:18.355996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68eb2e-093c-40ef-927f-b12a340260f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:18.356618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68eb2e-093c-40ef-927f-b12a340260f8 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:18.357262 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68eb2e-093c-40ef-927f-b12a340260f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:18.357574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68eb2e-093c-40ef-927f-b12a340260f8 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:18.358294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a68eb2e-093c-40ef-927f-b12a340260f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:18.361357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a68eb2e-093c-40ef-927f-b12a340260f8 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:18.470289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1230/2420] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:35:18 2022] POST /identity/v3/users => generated 309 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:18.477047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26b75d-d089-479c-959f-8d5e0e0bd409 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:18.508183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26b75d-d089-479c-959f-8d5e0e0bd409 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:18.515194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475af414-8e99-4894-aff4-b17093c05564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:18.516582 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475af414-8e99-4894-aff4-b17093c05564 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:18.517059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475af414-8e99-4894-aff4-b17093c05564 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:18.517687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475af414-8e99-4894-aff4-b17093c05564 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:18.518680 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475af414-8e99-4894-aff4-b17093c05564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:18.521083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475af414-8e99-4894-aff4-b17093c05564 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:18.543220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1191/2421] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:18.544704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68eb2e-093c-40ef-927f-b12a340260f8 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:18.558885 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68eb2e-093c-40ef-927f-b12a340260f8 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:18.565027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db2c3d-beec-4d97-b464-50633108428b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:18.565854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db2c3d-beec-4d97-b464-50633108428b admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:18.566206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db2c3d-beec-4d97-b464-50633108428b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:18.566591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db2c3d-beec-4d97-b464-50633108428b admin admin] PATH_INFO: `/v3/projects/f1bd9e18231141c684c2cb295e26da46/users/e13ae3b0f40a44aa8865f73dadb65203/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:18.588217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db2c3d-beec-4d97-b464-50633108428b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1bd9e18231141c684c2cb295e26da46, user_id=e13ae3b0f40a44aa8865f73dadb65203, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:18.591491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db2c3d-beec-4d97-b464-50633108428b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:18.704750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1231/2422] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:18 2022] PUT /identity/v3/projects/f1bd9e18231141c684c2cb295e26da46/users/e13ae3b0f40a44aa8865f73dadb65203/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:18.709979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475af414-8e99-4894-aff4-b17093c05564 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:18.724851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475af414-8e99-4894-aff4-b17093c05564 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:18.785592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:18.822365 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a77ab-c285-4aab-8fd3-f57293dd83e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:18.823390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a77ab-c285-4aab-8fd3-f57293dd83e3 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:18.823791 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a77ab-c285-4aab-8fd3-f57293dd83e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:18.824180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a77ab-c285-4aab-8fd3-f57293dd83e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:18.825324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a77ab-c285-4aab-8fd3-f57293dd83e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:18.828408 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a77ab-c285-4aab-8fd3-f57293dd83e3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:18.850018 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1192/2423] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:18.854933 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db2c3d-beec-4d97-b464-50633108428b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:18.873394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db2c3d-beec-4d97-b464-50633108428b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:18.883498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174310a-7cba-43f9-92d0-ec4071bb4686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:18.884810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174310a-7cba-43f9-92d0-ec4071bb4686 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:18.885300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174310a-7cba-43f9-92d0-ec4071bb4686 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:18.885780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174310a-7cba-43f9-92d0-ec4071bb4686 admin admin] PATH_INFO: `/v3/projects/f1bd9e18231141c684c2cb295e26da46/users/e13ae3b0f40a44aa8865f73dadb65203/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:18.889939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174310a-7cba-43f9-92d0-ec4071bb4686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1bd9e18231141c684c2cb295e26da46, user_id=e13ae3b0f40a44aa8865f73dadb65203, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:18.895474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174310a-7cba-43f9-92d0-ec4071bb4686 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:18.918731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1232/2424] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:18 2022] PUT /identity/v3/projects/f1bd9e18231141c684c2cb295e26da46/users/e13ae3b0f40a44aa8865f73dadb65203/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:35:18.932149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e34ae-e1e8-4fb6-b080-1d0155b24c68 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:18.933184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e34ae-e1e8-4fb6-b080-1d0155b24c68 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:18.933515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e34ae-e1e8-4fb6-b080-1d0155b24c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:18.971684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-926e34ae-e1e8-4fb6-b080-1d0155b24c68 None None] MFA Rules not processed for user `e13ae3b0f40a44aa8865f73dadb65203`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:18.973597 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-926e34ae-e1e8-4fb6-b080-1d0155b24c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:19.062850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1193/2425] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:18 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:19.078158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174310a-7cba-43f9-92d0-ec4071bb4686 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:19.081065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7174310a-7cba-43f9-92d0-ec4071bb4686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:19.107728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:19.159421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174310a-7cba-43f9-92d0-ec4071bb4686 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:19.163414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072a945-6739-46b2-a8f7-8ed20d6d78e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:19.164047 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072a945-6739-46b2-a8f7-8ed20d6d78e7 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:19.165438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072a945-6739-46b2-a8f7-8ed20d6d78e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:19.165438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072a945-6739-46b2-a8f7-8ed20d6d78e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:19.175459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072a945-6739-46b2-a8f7-8ed20d6d78e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:19.177102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072a945-6739-46b2-a8f7-8ed20d6d78e7 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:19.191124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1233/2426] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:19 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:35:19.435504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e34ae-e1e8-4fb6-b080-1d0155b24c68 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:19.533960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e34ae-e1e8-4fb6-b080-1d0155b24c68 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:19.543319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:19.545003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:19.545003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:19.545352 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] PATH_INFO: `/v3/users/e45da558ca4c4deb8755224624cfbec2` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:19.547411 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] RBAC: Authorizing `identity:delete_user(user_id=e45da558ca4c4deb8755224624cfbec2)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:19.551202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:19.761351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e45da558ca4c4deb8755224624cfbec2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:19.762454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e45da558ca4c4deb8755224624cfbec2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:19.763451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e45da558ca4c4deb8755224624cfbec2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:19.775438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e45da558ca4c4deb8755224624cfbec2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:19.783076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:19.794535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e45da558ca4c4deb8755224624cfbec2'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:19.879910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1194/2427] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:35:19 2022] DELETE /identity/v3/users/e45da558ca4c4deb8755224624cfbec2 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:19.888074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b296b-9f0c-449f-b973-cf78e39bb775 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:19.888712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b296b-9f0c-449f-b973-cf78e39bb775 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:19.889103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b296b-9f0c-449f-b973-cf78e39bb775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:19.928868 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-dd3b296b-9f0c-449f-b973-cf78e39bb775 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:19.931151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dd3b296b-9f0c-449f-b973-cf78e39bb775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:20.065808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1234/2428] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:20.075048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:20.077347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:20.151840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ca0e0-188e-4838-b849-7a5795c7e9da admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:20.156383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a855b78-22a6-40e4-a537-562e1d98a2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:20.157219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a855b78-22a6-40e4-a537-562e1d98a2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:20.157615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a855b78-22a6-40e4-a537-562e1d98a2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:20.157959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a855b78-22a6-40e4-a537-562e1d98a2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:20.172104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a855b78-22a6-40e4-a537-562e1d98a2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:20.173759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a855b78-22a6-40e4-a537-562e1d98a2b1 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:20.188181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1195/2429] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:35:20.701410 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b296b-9f0c-449f-b973-cf78e39bb775 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:20.703471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dd3b296b-9f0c-449f-b973-cf78e39bb775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:20.721777 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b296b-9f0c-449f-b973-cf78e39bb775 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:20.728026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:20.728682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:20.728966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:20.729235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] PATH_INFO: `/v3/projects/22862455fed848cb85fff5e74121b1fe` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:20.730395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22862455fed848cb85fff5e74121b1fe)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:20.732519 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:20.893181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22862455fed848cb85fff5e74121b1fe', 'request_id': 'req-dfe48e85-32f0-45fb-95a6-55828c6e78a7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:20.906537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22862455fed848cb85fff5e74121b1fe', 'request_id': 'req-dfe48e85-32f0-45fb-95a6-55828c6e78a7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:20.919110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22862455fed848cb85fff5e74121b1fe', 'request_id': 'req-dfe48e85-32f0-45fb-95a6-55828c6e78a7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:21.103610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] The token cache is being invalidate because project 22862455fed848cb85fff5e74121b1fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:35:21.104066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:21.104924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:21.110163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1235/2430] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:35:20 2022] DELETE /identity/v3/projects/22862455fed848cb85fff5e74121b1fe => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:21.111676 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a855b78-22a6-40e4-a537-562e1d98a2b1 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:21.124197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a855b78-22a6-40e4-a537-562e1d98a2b1 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:21.216304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:21.217392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:21.217949 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:21.218437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] PATH_INFO: `/v3/users/e13ae3b0f40a44aa8865f73dadb65203` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:21.220747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13ae3b0f40a44aa8865f73dadb65203)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:21.225754 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:21.418269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13ae3b0f40a44aa8865f73dadb65203'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:21.419004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13ae3b0f40a44aa8865f73dadb65203'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:21.419643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13ae3b0f40a44aa8865f73dadb65203'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:21.427821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13ae3b0f40a44aa8865f73dadb65203'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:21.446261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-a505d126-935d-495f-a6e2-be61b4d98622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13ae3b0f40a44aa8865f73dadb65203'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:21.534363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:21.540202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1196/2431] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:35:21 2022] DELETE /identity/v3/users/e13ae3b0f40a44aa8865f73dadb65203 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:22.025298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:22.027682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:22.044119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe48e85-32f0-45fb-95a6-55828c6e78a7 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:22.101289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:22.127270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:22.127984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:22.128266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:22.128545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] PATH_INFO: `/v3/projects/f1bd9e18231141c684c2cb295e26da46` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:22.129790 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1bd9e18231141c684c2cb295e26da46)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:22.132435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:22.276547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1bd9e18231141c684c2cb295e26da46', 'request_id': 'req-f717de9e-32db-4e0a-9690-00dbaa1e2aca'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:22.288788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1bd9e18231141c684c2cb295e26da46', 'request_id': 'req-f717de9e-32db-4e0a-9690-00dbaa1e2aca'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:22.299094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1bd9e18231141c684c2cb295e26da46', 'request_id': 'req-f717de9e-32db-4e0a-9690-00dbaa1e2aca'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:22.392468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] The token cache is being invalidate because project f1bd9e18231141c684c2cb295e26da46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:35:22.392775 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:22.393783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f717de9e-32db-4e0a-9690-00dbaa1e2aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:22.396183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1236/2432] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:35:22 2022] DELETE /identity/v3/projects/f1bd9e18231141c684c2cb295e26da46 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:22.401817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a9cb5-d83c-4f90-ba27-7df86a03b3f6 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:22.402710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a9cb5-d83c-4f90-ba27-7df86a03b3f6 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:22.402710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a9cb5-d83c-4f90-ba27-7df86a03b3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:22.425360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-5b4a9cb5-d83c-4f90-ba27-7df86a03b3f6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:22.426755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5b4a9cb5-d83c-4f90-ba27-7df86a03b3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:22.525119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1197/2433] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:35:22.535335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8710bcb-3ac8-4481-9594-f47bdd4497c7 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:22.535654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8710bcb-3ac8-4481-9594-f47bdd4497c7 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:22.536053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8710bcb-3ac8-4481-9594-f47bdd4497c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:22.561452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f8710bcb-3ac8-4481-9594-f47bdd4497c7 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:22.562820 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f8710bcb-3ac8-4481-9594-f47bdd4497c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:22.574401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1237/2434] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:22.582198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a9cb5-d83c-4f90-ba27-7df86a03b3f6 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:22.593342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a9cb5-d83c-4f90-ba27-7df86a03b3f6 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:22.598450 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bcc1f-75b0-4421-8235-5c5c566a0347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:22.599116 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bcc1f-75b0-4421-8235-5c5c566a0347 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:22.599434 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bcc1f-75b0-4421-8235-5c5c566a0347 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:22.599665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bcc1f-75b0-4421-8235-5c5c566a0347 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:22.601057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bcc1f-75b0-4421-8235-5c5c566a0347 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:35:22.601370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bcc1f-75b0-4421-8235-5c5c566a0347 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:22.603079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bcc1f-75b0-4421-8235-5c5c566a0347 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:22.617556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1198/2435] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:35:22.622408 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8710bcb-3ac8-4481-9594-f47bdd4497c7 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:22.633986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8710bcb-3ac8-4481-9594-f47bdd4497c7 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:22.639099 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb71278-9e61-44bc-a5cb-bed2454b579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:22.639722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb71278-9e61-44bc-a5cb-bed2454b579e admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:22.639993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb71278-9e61-44bc-a5cb-bed2454b579e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:22.640260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb71278-9e61-44bc-a5cb-bed2454b579e admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:22.640904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb71278-9e61-44bc-a5cb-bed2454b579e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:22.642881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb71278-9e61-44bc-a5cb-bed2454b579e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:22.775239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1238/2436] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:35:22 2022] POST /identity/v3/projects => generated 401 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:35:22.779912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bcc1f-75b0-4421-8235-5c5c566a0347 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:22.791801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bcc1f-75b0-4421-8235-5c5c566a0347 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:22.874035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7c25e-e1cf-42fd-a45a-a872cebeef6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:22.875715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7c25e-e1cf-42fd-a45a-a872cebeef6b admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:22.875715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7c25e-e1cf-42fd-a45a-a872cebeef6b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:22.875715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7c25e-e1cf-42fd-a45a-a872cebeef6b admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:22.876099 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7c25e-e1cf-42fd-a45a-a872cebeef6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:22.877801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7c25e-e1cf-42fd-a45a-a872cebeef6b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:23.042093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1199/2437] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:35:22 2022] POST /identity/v3/users => generated 301 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:35:23.046780 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb71278-9e61-44bc-a5cb-bed2454b579e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:23.059065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb71278-9e61-44bc-a5cb-bed2454b579e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:23.064167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2088-856f-4c9c-8ae5-7c05f839e91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:23.064807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2088-856f-4c9c-8ae5-7c05f839e91a admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:23.065081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2088-856f-4c9c-8ae5-7c05f839e91a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:23.065355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2088-856f-4c9c-8ae5-7c05f839e91a admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:23.066017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2088-856f-4c9c-8ae5-7c05f839e91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:23.067779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2088-856f-4c9c-8ae5-7c05f839e91a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:23.080452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1239/2438] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:23.085260 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7c25e-e1cf-42fd-a45a-a872cebeef6b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:23.096461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7c25e-e1cf-42fd-a45a-a872cebeef6b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:23.102432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56b809-458a-4980-aaf3-e4c804b33483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:23.108576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56b809-458a-4980-aaf3-e4c804b33483 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:23.108576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56b809-458a-4980-aaf3-e4c804b33483 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:23.108576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56b809-458a-4980-aaf3-e4c804b33483 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:23.108576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56b809-458a-4980-aaf3-e4c804b33483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:23.108576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56b809-458a-4980-aaf3-e4c804b33483 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:23.118784 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1200/2439] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:23.123406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2088-856f-4c9c-8ae5-7c05f839e91a admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:23.134549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2088-856f-4c9c-8ae5-7c05f839e91a admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:23.139565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d36a6-aa0b-41bc-a2eb-73c06fd2b545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:23.142965 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d36a6-aa0b-41bc-a2eb-73c06fd2b545 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:23.142965 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d36a6-aa0b-41bc-a2eb-73c06fd2b545 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:23.142965 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d36a6-aa0b-41bc-a2eb-73c06fd2b545 admin admin] PATH_INFO: `/v3/projects/370d1755e0fc45ec8f1ed73964edc396/users/64c6c8ce969f40b6a31393baf86a82bf/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:23.159330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d36a6-aa0b-41bc-a2eb-73c06fd2b545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=370d1755e0fc45ec8f1ed73964edc396, user_id=64c6c8ce969f40b6a31393baf86a82bf, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:23.162127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d36a6-aa0b-41bc-a2eb-73c06fd2b545 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:23.265365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1240/2440] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:23 2022] PUT /identity/v3/projects/370d1755e0fc45ec8f1ed73964edc396/users/64c6c8ce969f40b6a31393baf86a82bf/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:23.278859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f965d-1c97-4a1d-a16a-9e45935d0dd9 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:23.279400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f965d-1c97-4a1d-a16a-9e45935d0dd9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:23.279697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f965d-1c97-4a1d-a16a-9e45935d0dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:23.446378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d28f965d-1c97-4a1d-a16a-9e45935d0dd9 None None] MFA Rules not processed for user `64c6c8ce969f40b6a31393baf86a82bf`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:23.448143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d28f965d-1c97-4a1d-a16a-9e45935d0dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:23.524088 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:23.539423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1201/2441] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:23 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:23.544359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d36a6-aa0b-41bc-a2eb-73c06fd2b545 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:23.650323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d36a6-aa0b-41bc-a2eb-73c06fd2b545 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:23.656701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69c2fb-e3f7-4184-aa80-b24c70a0494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:23.657504 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69c2fb-e3f7-4184-aa80-b24c70a0494f admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:23.657796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69c2fb-e3f7-4184-aa80-b24c70a0494f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:23.658265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69c2fb-e3f7-4184-aa80-b24c70a0494f admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:23.659125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69c2fb-e3f7-4184-aa80-b24c70a0494f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:23.661288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69c2fb-e3f7-4184-aa80-b24c70a0494f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:23.680970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:23.766274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1241/2442] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:35:23 2022] POST /identity/v3/projects => generated 401 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:23.768823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f965d-1c97-4a1d-a16a-9e45935d0dd9 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:23.782685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f965d-1c97-4a1d-a16a-9e45935d0dd9 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:23.888056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b7fbe-4149-43fd-a556-62f69b929485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:23.888696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b7fbe-4149-43fd-a556-62f69b929485 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:23.888974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b7fbe-4149-43fd-a556-62f69b929485 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:23.889247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b7fbe-4149-43fd-a556-62f69b929485 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:23.889976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b7fbe-4149-43fd-a556-62f69b929485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:23.896148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b7fbe-4149-43fd-a556-62f69b929485 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:24.089853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1202/2443] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:35:23 2022] POST /identity/v3/users => generated 307 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:24.095522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69c2fb-e3f7-4184-aa80-b24c70a0494f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:24.109481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69c2fb-e3f7-4184-aa80-b24c70a0494f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:24.115062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587056-9f9c-4caa-94b3-6c2b0c929928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:24.116519 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587056-9f9c-4caa-94b3-6c2b0c929928 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:24.116801 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587056-9f9c-4caa-94b3-6c2b0c929928 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:24.117081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6587056-9f9c-4caa-94b3-6c2b0c929928 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:24.117747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6587056-9f9c-4caa-94b3-6c2b0c929928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:24.119487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6587056-9f9c-4caa-94b3-6c2b0c929928 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:24.135663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1242/2444] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:35:24.140648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b7fbe-4149-43fd-a556-62f69b929485 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:24.151913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b7fbe-4149-43fd-a556-62f69b929485 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:24.157256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c202de-2cf8-4821-ad2b-35f4881f301f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:24.157948 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c202de-2cf8-4821-ad2b-35f4881f301f admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:24.158225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c202de-2cf8-4821-ad2b-35f4881f301f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:24.158537 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c202de-2cf8-4821-ad2b-35f4881f301f admin admin] PATH_INFO: `/v3/projects/a26d1b8257724a789abcd9e123c13154/users/0588ecee2fb6472e886c6478421ffd44/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:24.180271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c202de-2cf8-4821-ad2b-35f4881f301f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26d1b8257724a789abcd9e123c13154, user_id=0588ecee2fb6472e886c6478421ffd44, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:24.183719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c202de-2cf8-4821-ad2b-35f4881f301f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:24.258397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1203/2445] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:24 2022] PUT /identity/v3/projects/a26d1b8257724a789abcd9e123c13154/users/0588ecee2fb6472e886c6478421ffd44/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:35:24.262660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587056-9f9c-4caa-94b3-6c2b0c929928 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:24.274899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6587056-9f9c-4caa-94b3-6c2b0c929928 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:24.352905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5e519-71a5-419c-adeb-c9d82e41a772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:24.353455 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5e519-71a5-419c-adeb-c9d82e41a772 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:24.353736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5e519-71a5-419c-adeb-c9d82e41a772 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:24.354005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5e519-71a5-419c-adeb-c9d82e41a772 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:24.354714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5e519-71a5-419c-adeb-c9d82e41a772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:24.356366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5e519-71a5-419c-adeb-c9d82e41a772 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:24.368274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1243/2446] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:24.372926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c202de-2cf8-4821-ad2b-35f4881f301f admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:24.388345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c202de-2cf8-4821-ad2b-35f4881f301f admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:24.395200 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750dfea-08ff-4275-bba6-46112dffb18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:24.396029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750dfea-08ff-4275-bba6-46112dffb18c admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:24.396413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750dfea-08ff-4275-bba6-46112dffb18c admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:24.396946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750dfea-08ff-4275-bba6-46112dffb18c admin admin] PATH_INFO: `/v3/projects/a26d1b8257724a789abcd9e123c13154/users/0588ecee2fb6472e886c6478421ffd44/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:24.399713 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750dfea-08ff-4275-bba6-46112dffb18c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26d1b8257724a789abcd9e123c13154, user_id=0588ecee2fb6472e886c6478421ffd44, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:24.403332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750dfea-08ff-4275-bba6-46112dffb18c admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:24.418220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1204/2447] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:24 2022] PUT /identity/v3/projects/a26d1b8257724a789abcd9e123c13154/users/0588ecee2fb6472e886c6478421ffd44/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:24.430936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f1897-b797-4068-9f14-341ab5399c7b None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:24.431358 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f1897-b797-4068-9f14-341ab5399c7b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:24.431730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f1897-b797-4068-9f14-341ab5399c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:24.480540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-fc0f1897-b797-4068-9f14-341ab5399c7b None None] MFA Rules not processed for user `0588ecee2fb6472e886c6478421ffd44`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:24.481836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fc0f1897-b797-4068-9f14-341ab5399c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:24.570416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1244/2448] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:24 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:24.581838 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750dfea-08ff-4275-bba6-46112dffb18c admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:24.583676 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8750dfea-08ff-4275-bba6-46112dffb18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:24.655900 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750dfea-08ff-4275-bba6-46112dffb18c admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:24.660653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425ebe0-807e-4186-a21b-cdff15f6d5f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:24.661516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425ebe0-807e-4186-a21b-cdff15f6d5f8 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:24.661928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425ebe0-807e-4186-a21b-cdff15f6d5f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:24.662285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425ebe0-807e-4186-a21b-cdff15f6d5f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:24.673885 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425ebe0-807e-4186-a21b-cdff15f6d5f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:24.675903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425ebe0-807e-4186-a21b-cdff15f6d5f8 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:24.688521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1205/2449] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:24 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:35:24.920120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f1897-b797-4068-9f14-341ab5399c7b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:24.933913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f1897-b797-4068-9f14-341ab5399c7b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:24.938838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb6bf8-0f6e-45fc-bf84-21ccc8baa6b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:24.939721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb6bf8-0f6e-45fc-bf84-21ccc8baa6b9 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:24.940048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb6bf8-0f6e-45fc-bf84-21ccc8baa6b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:24.940416 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb6bf8-0f6e-45fc-bf84-21ccc8baa6b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:25.018053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb6bf8-0f6e-45fc-bf84-21ccc8baa6b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:25.019885 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb6bf8-0f6e-45fc-bf84-21ccc8baa6b9 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:25.029523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:25.039426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1245/2450] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:24 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:35:30.532505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425ebe0-807e-4186-a21b-cdff15f6d5f8 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:30.602966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:30.617301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425ebe0-807e-4186-a21b-cdff15f6d5f8 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:30.622856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:30.623761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:30.624133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:30.624529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] PATH_INFO: `/v3/users/64c6c8ce969f40b6a31393baf86a82bf` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:30.626280 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c6c8ce969f40b6a31393baf86a82bf)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:30.628585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:30.932836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c6c8ce969f40b6a31393baf86a82bf'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:30.933617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c6c8ce969f40b6a31393baf86a82bf'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:30.934280 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c6c8ce969f40b6a31393baf86a82bf'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:30.944256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c6c8ce969f40b6a31393baf86a82bf'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:30.961586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c6c8ce969f40b6a31393baf86a82bf'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:31.043994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1206/2451] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:35:30 2022] DELETE /identity/v3/users/64c6c8ce969f40b6a31393baf86a82bf => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:31.048988 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6a3b6-c314-43f1-866c-5d2d1c40e9df None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:31.049845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6a3b6-c314-43f1-866c-5d2d1c40e9df None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:31.050061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6a3b6-c314-43f1-866c-5d2d1c40e9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:31.072203 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-07d6a3b6-c314-43f1-866c-5d2d1c40e9df None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:31.073544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-07d6a3b6-c314-43f1-866c-5d2d1c40e9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:31.144359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1246/2452] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:35:31.158152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:31.159707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:31.246366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1015c9e-f152-482f-ae23-70e2da1d6cc9 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:31.251391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca43c3-07a5-4e41-abf5-d9282b3b326e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:31.252273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca43c3-07a5-4e41-abf5-d9282b3b326e service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:31.252747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca43c3-07a5-4e41-abf5-d9282b3b326e service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:31.253377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca43c3-07a5-4e41-abf5-d9282b3b326e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:31.272879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca43c3-07a5-4e41-abf5-d9282b3b326e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:31.275220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca43c3-07a5-4e41-abf5-d9282b3b326e service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:31.295959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1207/2453] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:35:31.989237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6a3b6-c314-43f1-866c-5d2d1c40e9df None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:31.989237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-07d6a3b6-c314-43f1-866c-5d2d1c40e9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:32.010209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6a3b6-c314-43f1-866c-5d2d1c40e9df None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:32.016686 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:32.017472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:32.018075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:32.018433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] PATH_INFO: `/v3/projects/370d1755e0fc45ec8f1ed73964edc396` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:32.020153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=370d1755e0fc45ec8f1ed73964edc396)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:32.024414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:32.144531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370d1755e0fc45ec8f1ed73964edc396', 'request_id': 'req-9ebadb68-736b-4cd4-906a-e6ef2360fc24'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:32.153300 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370d1755e0fc45ec8f1ed73964edc396', 'request_id': 'req-9ebadb68-736b-4cd4-906a-e6ef2360fc24'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:32.163077 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370d1755e0fc45ec8f1ed73964edc396', 'request_id': 'req-9ebadb68-736b-4cd4-906a-e6ef2360fc24'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:32.235229 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] The token cache is being invalidate because project 370d1755e0fc45ec8f1ed73964edc396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:35:32.235522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:32.236132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:32.237739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1247/2454] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:35:31 2022] DELETE /identity/v3/projects/370d1755e0fc45ec8f1ed73964edc396 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:32.243014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca43c3-07a5-4e41-abf5-d9282b3b326e service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:32.260397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca43c3-07a5-4e41-abf5-d9282b3b326e service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:32.357264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:32.358281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:32.358778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:32.359207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] PATH_INFO: `/v3/users/0588ecee2fb6472e886c6478421ffd44` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:32.360975 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0588ecee2fb6472e886c6478421ffd44)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:32.364307 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:32.389943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:32.577663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0588ecee2fb6472e886c6478421ffd44'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:32.578707 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0588ecee2fb6472e886c6478421ffd44'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:32.579641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0588ecee2fb6472e886c6478421ffd44'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:32.591609 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0588ecee2fb6472e886c6478421ffd44'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:32.617146 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-38c55969-2f25-4ca1-9ed6-ee8ebd1bab1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0588ecee2fb6472e886c6478421ffd44'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:32.722262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1208/2455] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:35:32 2022] DELETE /identity/v3/users/0588ecee2fb6472e886c6478421ffd44 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:33.263565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:33.265913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:33.284809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebadb68-736b-4cd4-906a-e6ef2360fc24 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:33.367000 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:33.367756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:33.368085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:33.368428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] PATH_INFO: `/v3/projects/a26d1b8257724a789abcd9e123c13154` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:33.369753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26d1b8257724a789abcd9e123c13154)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:33.372236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:33.479592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:33.504148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26d1b8257724a789abcd9e123c13154', 'request_id': 'req-356f55a8-969b-4827-9a42-52e6ff94d692'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:33.514636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26d1b8257724a789abcd9e123c13154', 'request_id': 'req-356f55a8-969b-4827-9a42-52e6ff94d692'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:33.521025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26d1b8257724a789abcd9e123c13154', 'request_id': 'req-356f55a8-969b-4827-9a42-52e6ff94d692'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:33.632107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] The token cache is being invalidate because project a26d1b8257724a789abcd9e123c13154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:35:33.632400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:33.633334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-356f55a8-969b-4827-9a42-52e6ff94d692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:33.635002 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1248/2456] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:35:33 2022] DELETE /identity/v3/projects/a26d1b8257724a789abcd9e123c13154 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:35:33.640141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af4ba3-0d9f-4893-a048-8b2982322788 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:33.640907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af4ba3-0d9f-4893-a048-8b2982322788 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:33.640907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af4ba3-0d9f-4893-a048-8b2982322788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:33.666924 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-13af4ba3-0d9f-4893-a048-8b2982322788 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:33.668970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-13af4ba3-0d9f-4893-a048-8b2982322788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:33.756171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1209/2457] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:33.765766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fac2fd-e17b-4433-a05e-08b73f212fe2 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:33.766084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fac2fd-e17b-4433-a05e-08b73f212fe2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:33.766369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fac2fd-e17b-4433-a05e-08b73f212fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:33.789693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-63fac2fd-e17b-4433-a05e-08b73f212fe2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:33.791001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-63fac2fd-e17b-4433-a05e-08b73f212fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:33.799992 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1249/2458] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:35:33.805788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af4ba3-0d9f-4893-a048-8b2982322788 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:33.816215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af4ba3-0d9f-4893-a048-8b2982322788 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:33.821667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c1f77-d5ec-4f41-90d4-5e16697b4a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:33.822293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c1f77-d5ec-4f41-90d4-5e16697b4a09 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:33.822595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c1f77-d5ec-4f41-90d4-5e16697b4a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:33.822865 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c1f77-d5ec-4f41-90d4-5e16697b4a09 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:33.823530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c1f77-d5ec-4f41-90d4-5e16697b4a09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:35:33.823854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c1f77-d5ec-4f41-90d4-5e16697b4a09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:33.826011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c1f77-d5ec-4f41-90d4-5e16697b4a09 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:33.841455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1210/2459] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:35:33.845697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fac2fd-e17b-4433-a05e-08b73f212fe2 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:33.858146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fac2fd-e17b-4433-a05e-08b73f212fe2 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:33.867263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babe0f4-28b3-4cca-826b-509ec727cc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:33.868164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe0f4-28b3-4cca-826b-509ec727cc95 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:33.868655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe0f4-28b3-4cca-826b-509ec727cc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:33.869223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe0f4-28b3-4cca-826b-509ec727cc95 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:33.870384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babe0f4-28b3-4cca-826b-509ec727cc95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:33.874192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babe0f4-28b3-4cca-826b-509ec727cc95 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:33.996749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1250/2460] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:35:33 2022] POST /identity/v3/projects => generated 407 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:35:34.002222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c1f77-d5ec-4f41-90d4-5e16697b4a09 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:34.015956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c1f77-d5ec-4f41-90d4-5e16697b4a09 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:34.094332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f732b-a50a-42ab-9985-2919a80ab357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:34.095114 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f732b-a50a-42ab-9985-2919a80ab357 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:34.095492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f732b-a50a-42ab-9985-2919a80ab357 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:34.095867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f732b-a50a-42ab-9985-2919a80ab357 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:34.096694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f732b-a50a-42ab-9985-2919a80ab357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:34.099111 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f732b-a50a-42ab-9985-2919a80ab357 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:34.187919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1211/2461] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:35:34 2022] POST /identity/v3/users => generated 304 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:34.194273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babe0f4-28b3-4cca-826b-509ec727cc95 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:34.195702 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f732b-a50a-42ab-9985-2919a80ab357 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:34.197536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-692f732b-a50a-42ab-9985-2919a80ab357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:34.207766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babe0f4-28b3-4cca-826b-509ec727cc95 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:34.212131 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:34.214746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273a811-e4bc-4f52-ab07-82b3a5a5f7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:34.215462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273a811-e4bc-4f52-ab07-82b3a5a5f7ab admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:34.215861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273a811-e4bc-4f52-ab07-82b3a5a5f7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:34.216320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273a811-e4bc-4f52-ab07-82b3a5a5f7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:34.216962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273a811-e4bc-4f52-ab07-82b3a5a5f7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:34.219115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273a811-e4bc-4f52-ab07-82b3a5a5f7ab admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:34.234937 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1251/2462] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:34.240673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273a811-e4bc-4f52-ab07-82b3a5a5f7ab admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:34.259399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273a811-e4bc-4f52-ab07-82b3a5a5f7ab admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:34.266958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912d00c-812a-44b7-b66b-481b8d9ef6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:34.267930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912d00c-812a-44b7-b66b-481b8d9ef6df admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:34.268336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912d00c-812a-44b7-b66b-481b8d9ef6df admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:34.268794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912d00c-812a-44b7-b66b-481b8d9ef6df admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:34.269871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912d00c-812a-44b7-b66b-481b8d9ef6df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:34.274018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912d00c-812a-44b7-b66b-481b8d9ef6df admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:34.288903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f732b-a50a-42ab-9985-2919a80ab357 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:34.290640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1252/2463] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:35:34.294062 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b16bf-d295-4533-bd6a-a9174e9e0945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:34.300342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b16bf-d295-4533-bd6a-a9174e9e0945 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:34.300342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b16bf-d295-4533-bd6a-a9174e9e0945 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:34.300647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912d00c-812a-44b7-b66b-481b8d9ef6df admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:34.300803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b16bf-d295-4533-bd6a-a9174e9e0945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:34.300803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b32b16bf-d295-4533-bd6a-a9174e9e0945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:34.313596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912d00c-812a-44b7-b66b-481b8d9ef6df admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:34.320866 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7df1a8-0523-4fd7-b489-171af220a78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:34.321865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df1a8-0523-4fd7-b489-171af220a78d admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:34.322306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df1a8-0523-4fd7-b489-171af220a78d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:34.322737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df1a8-0523-4fd7-b489-171af220a78d admin admin] PATH_INFO: `/v3/projects/7d9ff336f5944e389400c04c21f47d55/users/7db98203fa8f41299d41c70ff631aa33/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:34.344434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7df1a8-0523-4fd7-b489-171af220a78d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9ff336f5944e389400c04c21f47d55, user_id=7db98203fa8f41299d41c70ff631aa33, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:34.348068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7df1a8-0523-4fd7-b489-171af220a78d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:34.389256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b16bf-d295-4533-bd6a-a9174e9e0945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:34.390799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b16bf-d295-4533-bd6a-a9174e9e0945 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:34.402578 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1212/2464] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:35:34.443036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1253/2465] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:34 2022] PUT /identity/v3/projects/7d9ff336f5944e389400c04c21f47d55/users/7db98203fa8f41299d41c70ff631aa33/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:34.456733 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca76d7-18c3-458a-808b-cf75020012af None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:34.456995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca76d7-18c3-458a-808b-cf75020012af None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:34.457276 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca76d7-18c3-458a-808b-cf75020012af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:34.510681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ebca76d7-18c3-458a-808b-cf75020012af None None] MFA Rules not processed for user `7db98203fa8f41299d41c70ff631aa33`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:34.512319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ebca76d7-18c3-458a-808b-cf75020012af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:34.610125 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1213/2466] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:34 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:35:34.612379 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7df1a8-0523-4fd7-b489-171af220a78d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:34.711254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7df1a8-0523-4fd7-b489-171af220a78d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:34.719873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21833d99-b402-4b4b-b02b-b982070f5332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:34.721151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21833d99-b402-4b4b-b02b-b982070f5332 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:34.721645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21833d99-b402-4b4b-b02b-b982070f5332 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:34.727848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21833d99-b402-4b4b-b02b-b982070f5332 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:34.727848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21833d99-b402-4b4b-b02b-b982070f5332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:34.727848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21833d99-b402-4b4b-b02b-b982070f5332 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:34.852393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1254/2467] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:35:34 2022] POST /identity/v3/projects => generated 407 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:34.853760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca76d7-18c3-458a-808b-cf75020012af None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:34.869283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca76d7-18c3-458a-808b-cf75020012af None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:34.956372 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:34.977515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c1ed6-508e-49ac-894d-5ca62f8e6f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:34.978233 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c1ed6-508e-49ac-894d-5ca62f8e6f8e admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:34.978650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c1ed6-508e-49ac-894d-5ca62f8e6f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:34.979080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c1ed6-508e-49ac-894d-5ca62f8e6f8e admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:34.979999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c1ed6-508e-49ac-894d-5ca62f8e6f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:34.982326 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c1ed6-508e-49ac-894d-5ca62f8e6f8e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:35.082094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1214/2468] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:35:34 2022] POST /identity/v3/users => generated 310 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:35:35.087697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21833d99-b402-4b4b-b02b-b982070f5332 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:35.100698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21833d99-b402-4b4b-b02b-b982070f5332 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:35.106051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a317c-2876-4117-a835-7dbcc873b32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:35.106846 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a317c-2876-4117-a835-7dbcc873b32c admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:35.108188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a317c-2876-4117-a835-7dbcc873b32c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:35.108188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a317c-2876-4117-a835-7dbcc873b32c admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:35.108460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a317c-2876-4117-a835-7dbcc873b32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:35.109990 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a317c-2876-4117-a835-7dbcc873b32c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:35.122790 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1255/2469] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:35:35.127364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c1ed6-508e-49ac-894d-5ca62f8e6f8e admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:35.138298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c1ed6-508e-49ac-894d-5ca62f8e6f8e admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:35.143854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14480621-d511-481c-bcba-5fcac9dcafbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:35.144649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14480621-d511-481c-bcba-5fcac9dcafbe admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:35.145043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14480621-d511-481c-bcba-5fcac9dcafbe admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:35.145411 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14480621-d511-481c-bcba-5fcac9dcafbe admin admin] PATH_INFO: `/v3/projects/8d5cf062ffbc45cf9d34dccb14cfa817/users/cb84cf4cf0a34e1d9bc607703a0e1a5a/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:35.164958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14480621-d511-481c-bcba-5fcac9dcafbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5cf062ffbc45cf9d34dccb14cfa817, user_id=cb84cf4cf0a34e1d9bc607703a0e1a5a, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:35.170638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14480621-d511-481c-bcba-5fcac9dcafbe admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:35.300756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1215/2470] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:35 2022] PUT /identity/v3/projects/8d5cf062ffbc45cf9d34dccb14cfa817/users/cb84cf4cf0a34e1d9bc607703a0e1a5a/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:35:35.305788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a317c-2876-4117-a835-7dbcc873b32c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:35.318206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a317c-2876-4117-a835-7dbcc873b32c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:35.330011 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:35.403497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397c5a3-6e61-49d2-8308-8b1085669173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:35.404578 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397c5a3-6e61-49d2-8308-8b1085669173 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:35.405153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397c5a3-6e61-49d2-8308-8b1085669173 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:35.405638 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397c5a3-6e61-49d2-8308-8b1085669173 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:35.406982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397c5a3-6e61-49d2-8308-8b1085669173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:35.410286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397c5a3-6e61-49d2-8308-8b1085669173 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:35.429153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1256/2471] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:35:35.434937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14480621-d511-481c-bcba-5fcac9dcafbe admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:35.448142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14480621-d511-481c-bcba-5fcac9dcafbe admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:35.453995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b3380-f568-4e9a-98ef-ca01ec65fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:35.454883 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b3380-f568-4e9a-98ef-ca01ec65fbdb admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:35.455239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b3380-f568-4e9a-98ef-ca01ec65fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:35.455692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b3380-f568-4e9a-98ef-ca01ec65fbdb admin admin] PATH_INFO: `/v3/projects/8d5cf062ffbc45cf9d34dccb14cfa817/users/cb84cf4cf0a34e1d9bc607703a0e1a5a/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:35.458435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b3380-f568-4e9a-98ef-ca01ec65fbdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5cf062ffbc45cf9d34dccb14cfa817, user_id=cb84cf4cf0a34e1d9bc607703a0e1a5a, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:35.461641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b3380-f568-4e9a-98ef-ca01ec65fbdb admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:35.477478 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1216/2472] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:35 2022] PUT /identity/v3/projects/8d5cf062ffbc45cf9d34dccb14cfa817/users/cb84cf4cf0a34e1d9bc607703a0e1a5a/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:35.489194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa323e-34bd-4b34-9e45-e569ef76526c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:35.489631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa323e-34bd-4b34-9e45-e569ef76526c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:35.490097 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa323e-34bd-4b34-9e45-e569ef76526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:35.526892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-98aa323e-34bd-4b34-9e45-e569ef76526c None None] MFA Rules not processed for user `cb84cf4cf0a34e1d9bc607703a0e1a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:35.528363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-98aa323e-34bd-4b34-9e45-e569ef76526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:35.632259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1257/2473] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:35 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:35.648297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b3380-f568-4e9a-98ef-ca01ec65fbdb admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:35.650686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-629b3380-f568-4e9a-98ef-ca01ec65fbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:35.738210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b3380-f568-4e9a-98ef-ca01ec65fbdb admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:35.745796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2863b-ea7a-4d33-a9fa-2c2237408d11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:35.746161 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2863b-ea7a-4d33-a9fa-2c2237408d11 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:35.746468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2863b-ea7a-4d33-a9fa-2c2237408d11 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:35.746852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2863b-ea7a-4d33-a9fa-2c2237408d11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:35.764000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2863b-ea7a-4d33-a9fa-2c2237408d11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:35.766884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2863b-ea7a-4d33-a9fa-2c2237408d11 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:35.781049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1217/2474] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:35 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:35:36.311683 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa323e-34bd-4b34-9e45-e569ef76526c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:36.328247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa323e-34bd-4b34-9e45-e569ef76526c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:36.332807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3d447-b992-4939-9cff-3df7159968f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:36.333737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3d447-b992-4939-9cff-3df7159968f0 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:36.334744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3d447-b992-4939-9cff-3df7159968f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:36.335124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3d447-b992-4939-9cff-3df7159968f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:36.421469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3d447-b992-4939-9cff-3df7159968f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:36.424332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3d447-b992-4939-9cff-3df7159968f0 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:36.442241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1258/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:36 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:35:45.775471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=48,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:46.433785 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=85 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:49.324989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2863b-ea7a-4d33-a9fa-2c2237408d11 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:49.397005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2863b-ea7a-4d33-a9fa-2c2237408d11 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:49.401636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:49.402259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:49.402561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:49.402840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] PATH_INFO: `/v3/users/7db98203fa8f41299d41c70ff631aa33` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:49.403958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db98203fa8f41299d41c70ff631aa33)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:49.405901 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:49.618075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db98203fa8f41299d41c70ff631aa33'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:49.623698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db98203fa8f41299d41c70ff631aa33'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:49.624846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db98203fa8f41299d41c70ff631aa33'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:49.640535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db98203fa8f41299d41c70ff631aa33'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:49.664923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db98203fa8f41299d41c70ff631aa33'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:49.738303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1218/2476] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:35:49 2022] DELETE /identity/v3/users/7db98203fa8f41299d41c70ff631aa33 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:49.744317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed74d0-524a-4782-9bde-fb37fb642812 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:49.745145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed74d0-524a-4782-9bde-fb37fb642812 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:49.745523 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed74d0-524a-4782-9bde-fb37fb642812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:49.772755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e7ed74d0-524a-4782-9bde-fb37fb642812 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:49.774674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e7ed74d0-524a-4782-9bde-fb37fb642812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:49.867697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1259/2477] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:49.878309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:49.883692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:49.949172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46059026-add0-440c-aaf6-ff45d2cb9c4f admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:49.953069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8be99c-9a82-4c11-81fd-64dad0486c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:49.953746 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8be99c-9a82-4c11-81fd-64dad0486c98 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:49.954058 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8be99c-9a82-4c11-81fd-64dad0486c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:49.954369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8be99c-9a82-4c11-81fd-64dad0486c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:49.967101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8be99c-9a82-4c11-81fd-64dad0486c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:49.969594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8be99c-9a82-4c11-81fd-64dad0486c98 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:49.980803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1219/2478] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:35:50.485080 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed74d0-524a-4782-9bde-fb37fb642812 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:50.487078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e7ed74d0-524a-4782-9bde-fb37fb642812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:50.509658 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed74d0-524a-4782-9bde-fb37fb642812 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:50.516009 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:50.516830 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:50.517202 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:50.517530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] PATH_INFO: `/v3/projects/7d9ff336f5944e389400c04c21f47d55` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:50.518954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9ff336f5944e389400c04c21f47d55)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:50.524462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:50.663119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9ff336f5944e389400c04c21f47d55', 'request_id': 'req-f8000772-5b82-4f1c-9d08-24e6bb742aea'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:50.670778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9ff336f5944e389400c04c21f47d55', 'request_id': 'req-f8000772-5b82-4f1c-9d08-24e6bb742aea'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:50.681111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9ff336f5944e389400c04c21f47d55', 'request_id': 'req-f8000772-5b82-4f1c-9d08-24e6bb742aea'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:50.781568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] The token cache is being invalidate because project 7d9ff336f5944e389400c04c21f47d55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:35:50.782297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:50.783014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:50.785272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1260/2479] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:35:50 2022] DELETE /identity/v3/projects/7d9ff336f5944e389400c04c21f47d55 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:50.790835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8be99c-9a82-4c11-81fd-64dad0486c98 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:50.801538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8be99c-9a82-4c11-81fd-64dad0486c98 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:50.824834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:50.873773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:50.874414 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:50.874749 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:50.875020 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] PATH_INFO: `/v3/users/cb84cf4cf0a34e1d9bc607703a0e1a5a` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:50.876274 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb84cf4cf0a34e1d9bc607703a0e1a5a)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:50.878243 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:51.151269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb84cf4cf0a34e1d9bc607703a0e1a5a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:51.152035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb84cf4cf0a34e1d9bc607703a0e1a5a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:51.152721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb84cf4cf0a34e1d9bc607703a0e1a5a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:51.162203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb84cf4cf0a34e1d9bc607703a0e1a5a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:51.178748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-16acfd14-0578-4af1-8bfa-9853b4388fb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb84cf4cf0a34e1d9bc607703a0e1a5a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:51.267123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1220/2480] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:35:50 2022] DELETE /identity/v3/users/cb84cf4cf0a34e1d9bc607703a0e1a5a => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:35:51.807749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:51.809877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:51.831273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8000772-5b82-4f1c-9d08-24e6bb742aea admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:51.918792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:51.919876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:51.920659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:51.921272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] PATH_INFO: `/v3/projects/8d5cf062ffbc45cf9d34dccb14cfa817` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:51.922724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d5cf062ffbc45cf9d34dccb14cfa817)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:51.925601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:52.041956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:52.061858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d5cf062ffbc45cf9d34dccb14cfa817', 'request_id': 'req-a8277b0c-f553-49e9-8655-97226fb0c5f5'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:52.069398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d5cf062ffbc45cf9d34dccb14cfa817', 'request_id': 'req-a8277b0c-f553-49e9-8655-97226fb0c5f5'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:52.075795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d5cf062ffbc45cf9d34dccb14cfa817', 'request_id': 'req-a8277b0c-f553-49e9-8655-97226fb0c5f5'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:52.163178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] The token cache is being invalidate because project 8d5cf062ffbc45cf9d34dccb14cfa817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:35:52.163498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:52.164186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a8277b0c-f553-49e9-8655-97226fb0c5f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:52.165800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1261/2481] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:35:51 2022] DELETE /identity/v3/projects/8d5cf062ffbc45cf9d34dccb14cfa817 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:52.171448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344879f-7933-4ac9-85bc-3a039c2ea438 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:52.175032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344879f-7933-4ac9-85bc-3a039c2ea438 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:52.175032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344879f-7933-4ac9-85bc-3a039c2ea438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:52.194967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3344879f-7933-4ac9-85bc-3a039c2ea438 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:52.196336 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3344879f-7933-4ac9-85bc-3a039c2ea438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:52.266047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1221/2482] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:52.275479 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411b2b7-3f11-487b-9d7d-b079f70be1e0 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:52.275904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411b2b7-3f11-487b-9d7d-b079f70be1e0 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:52.276290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411b2b7-3f11-487b-9d7d-b079f70be1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:52.299666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-7411b2b7-3f11-487b-9d7d-b079f70be1e0 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:52.300946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7411b2b7-3f11-487b-9d7d-b079f70be1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:52.309942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1262/2483] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:35:52.314793 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344879f-7933-4ac9-85bc-3a039c2ea438 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:52.325896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344879f-7933-4ac9-85bc-3a039c2ea438 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:52.330936 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599669e-06d1-40e6-9095-f5c2b3929d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:52.331556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599669e-06d1-40e6-9095-f5c2b3929d20 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:52.331831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599669e-06d1-40e6-9095-f5c2b3929d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:52.332096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599669e-06d1-40e6-9095-f5c2b3929d20 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:52.332757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599669e-06d1-40e6-9095-f5c2b3929d20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:35:52.333105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599669e-06d1-40e6-9095-f5c2b3929d20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:52.335614 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599669e-06d1-40e6-9095-f5c2b3929d20 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:52.350063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1222/2484] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:35:52.355449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411b2b7-3f11-487b-9d7d-b079f70be1e0 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:52.366678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411b2b7-3f11-487b-9d7d-b079f70be1e0 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:52.371897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc474e6-50b0-4ce6-a2c3-e1e62c824d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:52.372528 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc474e6-50b0-4ce6-a2c3-e1e62c824d8f admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:52.372816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc474e6-50b0-4ce6-a2c3-e1e62c824d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:52.373069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc474e6-50b0-4ce6-a2c3-e1e62c824d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:52.373721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc474e6-50b0-4ce6-a2c3-e1e62c824d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:52.375734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc474e6-50b0-4ce6-a2c3-e1e62c824d8f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:52.530987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1263/2485] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:35:52 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:35:52.538894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599669e-06d1-40e6-9095-f5c2b3929d20 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:52.558227 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599669e-06d1-40e6-9095-f5c2b3929d20 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:52.612516 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:52.675219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77fbd6-74c8-4be8-8c57-352a8aa08ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:52.676259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77fbd6-74c8-4be8-8c57-352a8aa08ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:52.676731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77fbd6-74c8-4be8-8c57-352a8aa08ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:52.678068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77fbd6-74c8-4be8-8c57-352a8aa08ae5 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:52.679309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77fbd6-74c8-4be8-8c57-352a8aa08ae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:52.683209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77fbd6-74c8-4be8-8c57-352a8aa08ae5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:52.860928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1223/2486] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:35:52 2022] POST /identity/v3/users => generated 291 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:52.865873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc474e6-50b0-4ce6-a2c3-e1e62c824d8f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:52.876815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc474e6-50b0-4ce6-a2c3-e1e62c824d8f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:52.882603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aceca2-84dc-443d-8e79-74dc26c400c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:52.883237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aceca2-84dc-443d-8e79-74dc26c400c1 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:52.883507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aceca2-84dc-443d-8e79-74dc26c400c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:52.883784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aceca2-84dc-443d-8e79-74dc26c400c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:52.884442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aceca2-84dc-443d-8e79-74dc26c400c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:52.886057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aceca2-84dc-443d-8e79-74dc26c400c1 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:52.898157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1264/2487] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:52.902811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77fbd6-74c8-4be8-8c57-352a8aa08ae5 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:52.914194 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77fbd6-74c8-4be8-8c57-352a8aa08ae5 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:52.919208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb68aa-77c8-4714-ab78-f07fbd0ee472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:52.919827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb68aa-77c8-4714-ab78-f07fbd0ee472 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:52.920107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb68aa-77c8-4714-ab78-f07fbd0ee472 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:52.920381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb68aa-77c8-4714-ab78-f07fbd0ee472 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:52.921033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb68aa-77c8-4714-ab78-f07fbd0ee472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:52.922864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb68aa-77c8-4714-ab78-f07fbd0ee472 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:52.934956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1224/2488] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:52.939756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aceca2-84dc-443d-8e79-74dc26c400c1 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:52.951755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aceca2-84dc-443d-8e79-74dc26c400c1 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:52.957067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133defa-f6b3-4f22-8149-0058ad119112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:52.957782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133defa-f6b3-4f22-8149-0058ad119112 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:52.958101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133defa-f6b3-4f22-8149-0058ad119112 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:52.959123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133defa-f6b3-4f22-8149-0058ad119112 admin admin] PATH_INFO: `/v3/projects/7d1af19c43f64e6ea856eb08145e577b/users/836575a416284494ae43d9e770a89a20/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:52.975363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133defa-f6b3-4f22-8149-0058ad119112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1af19c43f64e6ea856eb08145e577b, user_id=836575a416284494ae43d9e770a89a20, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:52.978044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133defa-f6b3-4f22-8149-0058ad119112 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:53.075670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1265/2489] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:52 2022] PUT /identity/v3/projects/7d1af19c43f64e6ea856eb08145e577b/users/836575a416284494ae43d9e770a89a20/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:53.086874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76480-d28c-4fa9-b3b5-6bead0a0a4bb None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:53.087154 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76480-d28c-4fa9-b3b5-6bead0a0a4bb None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:53.087431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76480-d28c-4fa9-b3b5-6bead0a0a4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:53.116410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-99b76480-d28c-4fa9-b3b5-6bead0a0a4bb None None] MFA Rules not processed for user `836575a416284494ae43d9e770a89a20`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:53.117541 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-99b76480-d28c-4fa9-b3b5-6bead0a0a4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:53.193404 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1225/2490] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:53.198245 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133defa-f6b3-4f22-8149-0058ad119112 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:53.268281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133defa-f6b3-4f22-8149-0058ad119112 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:53.273753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef11ca1-d081-46ab-97e9-7a81422bfce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:53.274386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef11ca1-d081-46ab-97e9-7a81422bfce0 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:53.274699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef11ca1-d081-46ab-97e9-7a81422bfce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:53.274971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef11ca1-d081-46ab-97e9-7a81422bfce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:53.275630 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef11ca1-d081-46ab-97e9-7a81422bfce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:53.277617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef11ca1-d081-46ab-97e9-7a81422bfce0 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:53.416959 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1266/2491] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:35:53 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:53.420890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b76480-d28c-4fa9-b3b5-6bead0a0a4bb None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:53.433068 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b76480-d28c-4fa9-b3b5-6bead0a0a4bb None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:53.495075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54f5a0-b052-4fa7-a5aa-680a74b7df54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:53.495738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54f5a0-b052-4fa7-a5aa-680a74b7df54 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:53.495976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54f5a0-b052-4fa7-a5aa-680a74b7df54 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:53.496303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54f5a0-b052-4fa7-a5aa-680a74b7df54 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:53.496961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54f5a0-b052-4fa7-a5aa-680a74b7df54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:53.498947 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54f5a0-b052-4fa7-a5aa-680a74b7df54 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:53.625358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1226/2492] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:35:53 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:53.626797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef11ca1-d081-46ab-97e9-7a81422bfce0 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:53.638619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef11ca1-d081-46ab-97e9-7a81422bfce0 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:53.643267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767127e-3275-4508-93dc-94be8bd2580b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:53.643880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767127e-3275-4508-93dc-94be8bd2580b admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:53.644157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767127e-3275-4508-93dc-94be8bd2580b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:53.644498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767127e-3275-4508-93dc-94be8bd2580b admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:53.645145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767127e-3275-4508-93dc-94be8bd2580b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:53.647604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767127e-3275-4508-93dc-94be8bd2580b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:53.659258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1267/2493] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:53.663876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54f5a0-b052-4fa7-a5aa-680a74b7df54 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:53.682169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54f5a0-b052-4fa7-a5aa-680a74b7df54 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:53.689520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16ae36-86cd-46b8-b8c7-2e4a21cb01a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:53.690814 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16ae36-86cd-46b8-b8c7-2e4a21cb01a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:53.691311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16ae36-86cd-46b8-b8c7-2e4a21cb01a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:53.691809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16ae36-86cd-46b8-b8c7-2e4a21cb01a8 admin admin] PATH_INFO: `/v3/projects/80eb7ac5a6a040caa4dee3d092c12f7e/users/bfd3404835a54ec1b4188608ae596f46/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:53.720819 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16ae36-86cd-46b8-b8c7-2e4a21cb01a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80eb7ac5a6a040caa4dee3d092c12f7e, user_id=bfd3404835a54ec1b4188608ae596f46, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:53.726106 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16ae36-86cd-46b8-b8c7-2e4a21cb01a8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:53.809671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1227/2494] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:53 2022] PUT /identity/v3/projects/80eb7ac5a6a040caa4dee3d092c12f7e/users/bfd3404835a54ec1b4188608ae596f46/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:53.814236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767127e-3275-4508-93dc-94be8bd2580b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:53.826831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767127e-3275-4508-93dc-94be8bd2580b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:53.848928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:53.896298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e863893-5943-4296-87ad-49bdadb569cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:53.898444 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e863893-5943-4296-87ad-49bdadb569cf admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:53.898942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e863893-5943-4296-87ad-49bdadb569cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:53.899428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e863893-5943-4296-87ad-49bdadb569cf admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:53.900584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e863893-5943-4296-87ad-49bdadb569cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:53.903915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e863893-5943-4296-87ad-49bdadb569cf admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:53.921423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1268/2495] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:35:53.925619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16ae36-86cd-46b8-b8c7-2e4a21cb01a8 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:53.931921 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:53.937576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16ae36-86cd-46b8-b8c7-2e4a21cb01a8 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:53.943165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a0382-6abf-4cd7-92bb-ef22a569b22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:53.944133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a0382-6abf-4cd7-92bb-ef22a569b22a admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:53.944445 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a0382-6abf-4cd7-92bb-ef22a569b22a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:53.944712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a0382-6abf-4cd7-92bb-ef22a569b22a admin admin] PATH_INFO: `/v3/projects/80eb7ac5a6a040caa4dee3d092c12f7e/users/bfd3404835a54ec1b4188608ae596f46/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:53.946756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a0382-6abf-4cd7-92bb-ef22a569b22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=80eb7ac5a6a040caa4dee3d092c12f7e, user_id=bfd3404835a54ec1b4188608ae596f46, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:53.949689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a0382-6abf-4cd7-92bb-ef22a569b22a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:53.960581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1228/2496] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:53 2022] PUT /identity/v3/projects/80eb7ac5a6a040caa4dee3d092c12f7e/users/bfd3404835a54ec1b4188608ae596f46/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:35:53.972025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f837d-0c53-400b-aac4-7145c6735e39 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:53.972353 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f837d-0c53-400b-aac4-7145c6735e39 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:53.972749 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f837d-0c53-400b-aac4-7145c6735e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:54.002297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-3c5f837d-0c53-400b-aac4-7145c6735e39 None None] MFA Rules not processed for user `bfd3404835a54ec1b4188608ae596f46`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:54.003568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3c5f837d-0c53-400b-aac4-7145c6735e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:54.115123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1269/2497] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:54.130322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a0382-6abf-4cd7-92bb-ef22a569b22a admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:54.133183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ea7a0382-6abf-4cd7-92bb-ef22a569b22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:54.210901 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a0382-6abf-4cd7-92bb-ef22a569b22a admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:54.215107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202fd7c-01f7-4f4a-b98e-8491888549a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:54.215761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202fd7c-01f7-4f4a-b98e-8491888549a1 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:54.216039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202fd7c-01f7-4f4a-b98e-8491888549a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:54.216310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202fd7c-01f7-4f4a-b98e-8491888549a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:54.227332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202fd7c-01f7-4f4a-b98e-8491888549a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:54.228877 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202fd7c-01f7-4f4a-b98e-8491888549a1 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:54.240297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1229/2498] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:35:54.384286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f837d-0c53-400b-aac4-7145c6735e39 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:54.394276 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f837d-0c53-400b-aac4-7145c6735e39 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:54.398772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02557535-fa09-4d4d-b368-c0a6d683d838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:54.399832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02557535-fa09-4d4d-b368-c0a6d683d838 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:54.399832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02557535-fa09-4d4d-b368-c0a6d683d838 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:54.400133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02557535-fa09-4d4d-b368-c0a6d683d838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:54.481461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02557535-fa09-4d4d-b368-c0a6d683d838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:54.482921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02557535-fa09-4d4d-b368-c0a6d683d838 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:54.493734 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1270/2499] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:35:55.120787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202fd7c-01f7-4f4a-b98e-8491888549a1 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:55.197517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202fd7c-01f7-4f4a-b98e-8491888549a1 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:55.202422 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:55.203105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:55.203378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:55.203646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] PATH_INFO: `/v3/users/836575a416284494ae43d9e770a89a20` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:55.204833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] RBAC: Authorizing `identity:delete_user(user_id=836575a416284494ae43d9e770a89a20)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:55.206842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:55.417719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '836575a416284494ae43d9e770a89a20'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:55.418876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '836575a416284494ae43d9e770a89a20'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:55.419584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '836575a416284494ae43d9e770a89a20'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:55.429052 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '836575a416284494ae43d9e770a89a20'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:55.443849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '836575a416284494ae43d9e770a89a20'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:55.555893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1230/2500] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:35:55 2022] DELETE /identity/v3/users/836575a416284494ae43d9e770a89a20 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:55.561969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8bb94-e4a0-483d-ace0-44c618b28ed4 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:55.562852 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8bb94-e4a0-483d-ace0-44c618b28ed4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:55.562852 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8bb94-e4a0-483d-ace0-44c618b28ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:55.586604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-40c8bb94-e4a0-483d-ace0-44c618b28ed4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:55.588770 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-40c8bb94-e4a0-483d-ace0-44c618b28ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:55.666791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:55.690441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1271/2501] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:35:55.704457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:55.705912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:55.722727 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:55.778297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861894a8-299f-4f93-b977-7c29595ae01d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:55.782927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb393305-a3dc-42bc-9a41-40c051384d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:55.783681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393305-a3dc-42bc-9a41-40c051384d51 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:55.784082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393305-a3dc-42bc-9a41-40c051384d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:55.784393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393305-a3dc-42bc-9a41-40c051384d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:55.797549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb393305-a3dc-42bc-9a41-40c051384d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:55.804113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb393305-a3dc-42bc-9a41-40c051384d51 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:55.820024 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1231/2502] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:35:56.394400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8bb94-e4a0-483d-ace0-44c618b28ed4 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:56.400018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-40c8bb94-e4a0-483d-ace0-44c618b28ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:56.419648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8bb94-e4a0-483d-ace0-44c618b28ed4 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:56.424951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:56.425606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:56.425882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:56.426154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] PATH_INFO: `/v3/projects/7d1af19c43f64e6ea856eb08145e577b` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:56.427972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1af19c43f64e6ea856eb08145e577b)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:56.430407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:56.547359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1af19c43f64e6ea856eb08145e577b', 'request_id': 'req-642e57eb-108d-4292-a7bc-0fbbea32a018'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:56.556269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1af19c43f64e6ea856eb08145e577b', 'request_id': 'req-642e57eb-108d-4292-a7bc-0fbbea32a018'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:56.563247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1af19c43f64e6ea856eb08145e577b', 'request_id': 'req-642e57eb-108d-4292-a7bc-0fbbea32a018'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:56.652752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] The token cache is being invalidate because project 7d1af19c43f64e6ea856eb08145e577b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:35:56.653089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:56.653783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:56.655939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1272/2503] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:35:56 2022] DELETE /identity/v3/projects/7d1af19c43f64e6ea856eb08145e577b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:35:56.660522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb393305-a3dc-42bc-9a41-40c051384d51 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:56.671357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb393305-a3dc-42bc-9a41-40c051384d51 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:56.741407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:56.742186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:56.742613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:56.743013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] PATH_INFO: `/v3/users/bfd3404835a54ec1b4188608ae596f46` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:56.744429 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd3404835a54ec1b4188608ae596f46)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:56.746523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:57.010709 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd3404835a54ec1b4188608ae596f46'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:57.011310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd3404835a54ec1b4188608ae596f46'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:57.011957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd3404835a54ec1b4188608ae596f46'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:57.021821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd3404835a54ec1b4188608ae596f46'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:57.041499 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-63281885-0656-448d-991c-498aa13b7b89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd3404835a54ec1b4188608ae596f46'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:57.172447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1232/2504] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:35:56 2022] DELETE /identity/v3/users/bfd3404835a54ec1b4188608ae596f46 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:57.701500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:57.707793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:57.717189 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e57eb-108d-4292-a7bc-0fbbea32a018 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:57.797597 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:57.798253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:57.798698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:57.799089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] PATH_INFO: `/v3/projects/80eb7ac5a6a040caa4dee3d092c12f7e` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:57.800346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] RBAC: Authorizing `identity:delete_project(project_id=80eb7ac5a6a040caa4dee3d092c12f7e)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:57.803107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:57.945747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80eb7ac5a6a040caa4dee3d092c12f7e', 'request_id': 'req-14661586-26a8-4842-8a27-ce9899b0e33c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:57.953720 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80eb7ac5a6a040caa4dee3d092c12f7e', 'request_id': 'req-14661586-26a8-4842-8a27-ce9899b0e33c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:57.962584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80eb7ac5a6a040caa4dee3d092c12f7e', 'request_id': 'req-14661586-26a8-4842-8a27-ce9899b0e33c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:58.035670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:58.042036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] The token cache is being invalidate because project 80eb7ac5a6a040caa4dee3d092c12f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:35:58.047690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:58.047690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14661586-26a8-4842-8a27-ce9899b0e33c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:58.047690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1273/2505] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:35:57 2022] DELETE /identity/v3/projects/80eb7ac5a6a040caa4dee3d092c12f7e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:35:58.050691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2fec7-01a5-463d-82c8-b4a6ecdfad5b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:58.051501 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2fec7-01a5-463d-82c8-b4a6ecdfad5b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:58.051668 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2fec7-01a5-463d-82c8-b4a6ecdfad5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:58.075436 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-6fd2fec7-01a5-463d-82c8-b4a6ecdfad5b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:58.076768 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-6fd2fec7-01a5-463d-82c8-b4a6ecdfad5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:58.112631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:58.148391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1233/2506] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:58.158158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8c3c3-54dd-4a78-942d-0a9b43424859 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:58.158709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8c3c3-54dd-4a78-942d-0a9b43424859 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:58.159242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8c3c3-54dd-4a78-942d-0a9b43424859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:58.195908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-3fe8c3c3-54dd-4a78-942d-0a9b43424859 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:58.197694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3fe8c3c3-54dd-4a78-942d-0a9b43424859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:58.211845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1274/2507] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:58.216899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2fec7-01a5-463d-82c8-b4a6ecdfad5b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:58.229679 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2fec7-01a5-463d-82c8-b4a6ecdfad5b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:58.234910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa950ba-42d9-42cf-92ae-37d028679f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:58.235630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa950ba-42d9-42cf-92ae-37d028679f9a admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:58.235878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa950ba-42d9-42cf-92ae-37d028679f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:58.236144 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa950ba-42d9-42cf-92ae-37d028679f9a admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:58.236869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa950ba-42d9-42cf-92ae-37d028679f9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:35:58.237152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa950ba-42d9-42cf-92ae-37d028679f9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:58.238888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa950ba-42d9-42cf-92ae-37d028679f9a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:58.254671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1234/2508] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:35:58.260287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8c3c3-54dd-4a78-942d-0a9b43424859 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:58.271314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8c3c3-54dd-4a78-942d-0a9b43424859 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:58.277327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a669f5-52a4-4abc-ad09-987b3e5652de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:58.278006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a669f5-52a4-4abc-ad09-987b3e5652de admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:58.278374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a669f5-52a4-4abc-ad09-987b3e5652de admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:58.278753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a669f5-52a4-4abc-ad09-987b3e5652de admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:58.279453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a669f5-52a4-4abc-ad09-987b3e5652de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:58.281480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a669f5-52a4-4abc-ad09-987b3e5652de admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:58.394360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1275/2509] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:35:58 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:35:58.400021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa950ba-42d9-42cf-92ae-37d028679f9a admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:58.411989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa950ba-42d9-42cf-92ae-37d028679f9a admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:58.497460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791541c3-ed9c-405b-a349-d7d713d0c662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:58.498098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791541c3-ed9c-405b-a349-d7d713d0c662 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:58.498377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791541c3-ed9c-405b-a349-d7d713d0c662 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:58.498695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791541c3-ed9c-405b-a349-d7d713d0c662 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:58.499342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791541c3-ed9c-405b-a349-d7d713d0c662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:58.501491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791541c3-ed9c-405b-a349-d7d713d0c662 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:58.650904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1235/2510] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:35:58 2022] POST /identity/v3/users => generated 287 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:35:58.662001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a669f5-52a4-4abc-ad09-987b3e5652de admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:58.668094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a669f5-52a4-4abc-ad09-987b3e5652de admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:58.673492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8728c-cb28-46da-9af1-ab6d7aca5aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:58.674222 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8728c-cb28-46da-9af1-ab6d7aca5aaf admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:58.674582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8728c-cb28-46da-9af1-ab6d7aca5aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:58.674887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8728c-cb28-46da-9af1-ab6d7aca5aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:58.675634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8728c-cb28-46da-9af1-ab6d7aca5aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:58.677330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8728c-cb28-46da-9af1-ab6d7aca5aaf admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:58.689311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1276/2511] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:58.696494 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791541c3-ed9c-405b-a349-d7d713d0c662 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:58.714331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791541c3-ed9c-405b-a349-d7d713d0c662 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:58.722193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773baf7-260a-4841-9c1c-27cf7cdc10e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:58.723206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773baf7-260a-4841-9c1c-27cf7cdc10e3 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:58.724106 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773baf7-260a-4841-9c1c-27cf7cdc10e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:58.724682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773baf7-260a-4841-9c1c-27cf7cdc10e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:58.726051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773baf7-260a-4841-9c1c-27cf7cdc10e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:58.728610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773baf7-260a-4841-9c1c-27cf7cdc10e3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:58.748613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1236/2512] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:35:58.754356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8728c-cb28-46da-9af1-ab6d7aca5aaf admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:58.767133 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8728c-cb28-46da-9af1-ab6d7aca5aaf admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:58.773107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda9e8a-f753-427b-9925-2fb1b2a3bbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:58.773839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda9e8a-f753-427b-9925-2fb1b2a3bbe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:58.774156 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda9e8a-f753-427b-9925-2fb1b2a3bbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:58.774468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda9e8a-f753-427b-9925-2fb1b2a3bbe7 admin admin] PATH_INFO: `/v3/projects/4f031796779b411c9612d3f66a9f1144/users/b564f093e3e6450faa14f4179b39a312/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:58.793254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda9e8a-f753-427b-9925-2fb1b2a3bbe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f031796779b411c9612d3f66a9f1144, user_id=b564f093e3e6450faa14f4179b39a312, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:58.796014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda9e8a-f753-427b-9925-2fb1b2a3bbe7 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:58.970068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1277/2513] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:35:58 2022] PUT /identity/v3/projects/4f031796779b411c9612d3f66a9f1144/users/b564f093e3e6450faa14f4179b39a312/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:58.981533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a79b2-445f-4bfc-9b82-f46158c126aa None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:58.981869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a79b2-445f-4bfc-9b82-f46158c126aa None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:58.982210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a79b2-445f-4bfc-9b82-f46158c126aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:59.018513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ed5a79b2-445f-4bfc-9b82-f46158c126aa None None] MFA Rules not processed for user `b564f093e3e6450faa14f4179b39a312`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:59.019804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ed5a79b2-445f-4bfc-9b82-f46158c126aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:59.099651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1237/2514] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:59.116969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda9e8a-f753-427b-9925-2fb1b2a3bbe7 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:59.118835 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-cfda9e8a-f753-427b-9925-2fb1b2a3bbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:59.184826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda9e8a-f753-427b-9925-2fb1b2a3bbe7 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:59.188926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043f5f4-6c59-4aa4-a53a-a45b9de01051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:59.189560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043f5f4-6c59-4aa4-a53a-a45b9de01051 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:59.189839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043f5f4-6c59-4aa4-a53a-a45b9de01051 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:59.190115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043f5f4-6c59-4aa4-a53a-a45b9de01051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:59.200148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043f5f4-6c59-4aa4-a53a-a45b9de01051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:59.201747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043f5f4-6c59-4aa4-a53a-a45b9de01051 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:59.213075 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1278/2515] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:35:59.243273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a79b2-445f-4bfc-9b82-f46158c126aa None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:59.290670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:35:59.329702 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a79b2-445f-4bfc-9b82-f46158c126aa None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:59.336017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:59.336724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:59.337011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:59.337281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] PATH_INFO: `/v3/users/b564f093e3e6450faa14f4179b39a312` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:59.338526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b564f093e3e6450faa14f4179b39a312)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:59.340502 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:59.626831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b564f093e3e6450faa14f4179b39a312'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:59.627090 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b564f093e3e6450faa14f4179b39a312'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:59.628043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b564f093e3e6450faa14f4179b39a312'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:59.637610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b564f093e3e6450faa14f4179b39a312'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:59.651632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b564f093e3e6450faa14f4179b39a312'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:35:59.746665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1238/2516] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:35:59 2022] DELETE /identity/v3/users/b564f093e3e6450faa14f4179b39a312 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:35:59.751775 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae82587-046b-4146-bae9-d342f812b35b None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:59.752254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae82587-046b-4146-bae9-d342f812b35b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:59.752626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae82587-046b-4146-bae9-d342f812b35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:59.775604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-5ae82587-046b-4146-bae9-d342f812b35b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:35:59.776847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5ae82587-046b-4146-bae9-d342f812b35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:59.846774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1279/2517] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:35:59.858663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:35:59.860336 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:35:59.932147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1219384-9054-4476-942b-510d4045b123 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:35:59.945850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c60549-39ad-4556-8f41-80360ecf9836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:35:59.945850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60549-39ad-4556-8f41-80360ecf9836 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:35:59.945850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60549-39ad-4556-8f41-80360ecf9836 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:35:59.945850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60549-39ad-4556-8f41-80360ecf9836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:35:59.950510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c60549-39ad-4556-8f41-80360ecf9836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:35:59.952135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c60549-39ad-4556-8f41-80360ecf9836 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:35:59.964613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1239/2518] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:35:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:36:00.503177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae82587-046b-4146-bae9-d342f812b35b None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:00.503177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5ae82587-046b-4146-bae9-d342f812b35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:00.523910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae82587-046b-4146-bae9-d342f812b35b None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:00.531550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:00.532463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:00.532895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:00.533267 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] PATH_INFO: `/v3/projects/4f031796779b411c9612d3f66a9f1144` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:00.535193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f031796779b411c9612d3f66a9f1144)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:00.538426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:00.642158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:00.657259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f031796779b411c9612d3f66a9f1144', 'request_id': 'req-28c7439b-0aaf-472d-8757-14b19b7e6917'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:00.665284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f031796779b411c9612d3f66a9f1144', 'request_id': 'req-28c7439b-0aaf-472d-8757-14b19b7e6917'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:00.672610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f031796779b411c9612d3f66a9f1144', 'request_id': 'req-28c7439b-0aaf-472d-8757-14b19b7e6917'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:00.766552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] The token cache is being invalidate because project 4f031796779b411c9612d3f66a9f1144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:36:00.766810 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:00.767694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-28c7439b-0aaf-472d-8757-14b19b7e6917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:00.769360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1280/2519] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:36:00 2022] DELETE /identity/v3/projects/4f031796779b411c9612d3f66a9f1144 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:00.775269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04363a-0072-43cd-9eb6-0bb616fd084c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:00.775991 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04363a-0072-43cd-9eb6-0bb616fd084c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:00.775991 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04363a-0072-43cd-9eb6-0bb616fd084c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:00.798815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-2f04363a-0072-43cd-9eb6-0bb616fd084c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:00.800138 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-2f04363a-0072-43cd-9eb6-0bb616fd084c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:00.858701 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:00.875259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1240/2520] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:00.946142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d313dc-d666-4d2b-a493-cd9c1a7d909f None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:00.947027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d313dc-d666-4d2b-a493-cd9c1a7d909f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:00.947211 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d313dc-d666-4d2b-a493-cd9c1a7d909f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:00.969475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c6d313dc-d666-4d2b-a493-cd9c1a7d909f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:00.970831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c6d313dc-d666-4d2b-a493-cd9c1a7d909f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:00.979446 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1281/2521] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:00.984017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04363a-0072-43cd-9eb6-0bb616fd084c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:00.995197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04363a-0072-43cd-9eb6-0bb616fd084c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:01.000225 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22099b53-5f1e-4dd9-91e9-0ff11cffe535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:01.000852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22099b53-5f1e-4dd9-91e9-0ff11cffe535 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:01.001127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22099b53-5f1e-4dd9-91e9-0ff11cffe535 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:01.001409 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22099b53-5f1e-4dd9-91e9-0ff11cffe535 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:01.002621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22099b53-5f1e-4dd9-91e9-0ff11cffe535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:36:01.002942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22099b53-5f1e-4dd9-91e9-0ff11cffe535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:01.004579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22099b53-5f1e-4dd9-91e9-0ff11cffe535 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:01.019267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1241/2522] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:36:01.024265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d313dc-d666-4d2b-a493-cd9c1a7d909f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:01.034421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d313dc-d666-4d2b-a493-cd9c1a7d909f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:01.040767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dce0c9-d4d7-4b7c-a992-93a0f4302fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:01.041401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dce0c9-d4d7-4b7c-a992-93a0f4302fbf admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:01.041671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dce0c9-d4d7-4b7c-a992-93a0f4302fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:01.041941 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dce0c9-d4d7-4b7c-a992-93a0f4302fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:01.042632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dce0c9-d4d7-4b7c-a992-93a0f4302fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:01.044582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dce0c9-d4d7-4b7c-a992-93a0f4302fbf admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:01.147525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1282/2523] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:36:01 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:36:01.153135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22099b53-5f1e-4dd9-91e9-0ff11cffe535 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:01.164113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22099b53-5f1e-4dd9-91e9-0ff11cffe535 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:01.234918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40500-f3f3-4990-a94b-9e444c2a9422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:01.235553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40500-f3f3-4990-a94b-9e444c2a9422 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:01.235826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40500-f3f3-4990-a94b-9e444c2a9422 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:01.236090 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40500-f3f3-4990-a94b-9e444c2a9422 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:01.236731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb40500-f3f3-4990-a94b-9e444c2a9422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:01.238855 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb40500-f3f3-4990-a94b-9e444c2a9422 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:01.355374 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1242/2524] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:36:01 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:36:01.359691 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dce0c9-d4d7-4b7c-a992-93a0f4302fbf admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:01.371357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dce0c9-d4d7-4b7c-a992-93a0f4302fbf admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:01.376396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105df483-7acb-4ba8-a7a2-69945117d8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:01.377023 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105df483-7acb-4ba8-a7a2-69945117d8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:01.377299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105df483-7acb-4ba8-a7a2-69945117d8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:01.377569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105df483-7acb-4ba8-a7a2-69945117d8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:01.378218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105df483-7acb-4ba8-a7a2-69945117d8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:01.380588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105df483-7acb-4ba8-a7a2-69945117d8e9 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:01.392218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1283/2525] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:36:01.397556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40500-f3f3-4990-a94b-9e444c2a9422 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:01.408390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40500-f3f3-4990-a94b-9e444c2a9422 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:01.414141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049be89-60f7-4d92-8dc0-cbba20b09132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:01.414806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049be89-60f7-4d92-8dc0-cbba20b09132 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:01.415080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049be89-60f7-4d92-8dc0-cbba20b09132 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:01.415349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049be89-60f7-4d92-8dc0-cbba20b09132 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:01.416010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049be89-60f7-4d92-8dc0-cbba20b09132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:01.417650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049be89-60f7-4d92-8dc0-cbba20b09132 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:01.429602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1243/2526] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:36:01.434797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105df483-7acb-4ba8-a7a2-69945117d8e9 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:01.453386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105df483-7acb-4ba8-a7a2-69945117d8e9 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:01.461893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77268255-a8c0-472d-8453-f7d0bd676def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:01.463095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268255-a8c0-472d-8453-f7d0bd676def admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:01.463588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268255-a8c0-472d-8453-f7d0bd676def admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:01.464100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268255-a8c0-472d-8453-f7d0bd676def admin admin] PATH_INFO: `/v3/projects/8d345a815fcf4d4bb57d98c8dbd60a09/users/95439da9d9c545d5af8624feaf9dffd4/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:01.492435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77268255-a8c0-472d-8453-f7d0bd676def admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d345a815fcf4d4bb57d98c8dbd60a09, user_id=95439da9d9c545d5af8624feaf9dffd4, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:01.497943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77268255-a8c0-472d-8453-f7d0bd676def admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:01.590292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1284/2527] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:36:01 2022] PUT /identity/v3/projects/8d345a815fcf4d4bb57d98c8dbd60a09/users/95439da9d9c545d5af8624feaf9dffd4/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:01.601043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a7b10-d5e4-43c3-87b8-a23ceebfefd8 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:01.601316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a7b10-d5e4-43c3-87b8-a23ceebfefd8 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:01.601636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a7b10-d5e4-43c3-87b8-a23ceebfefd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:01.635552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-606a7b10-d5e4-43c3-87b8-a23ceebfefd8 None None] MFA Rules not processed for user `95439da9d9c545d5af8624feaf9dffd4`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:01.636954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-606a7b10-d5e4-43c3-87b8-a23ceebfefd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:01.712667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1244/2528] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:01.717505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77268255-a8c0-472d-8453-f7d0bd676def admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:01.792164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77268255-a8c0-472d-8453-f7d0bd676def admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:01.800550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee086f-ae5e-43f7-952f-bf8ce693bbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:01.801842 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee086f-ae5e-43f7-952f-bf8ce693bbb1 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:01.803693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee086f-ae5e-43f7-952f-bf8ce693bbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:01.803693 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee086f-ae5e-43f7-952f-bf8ce693bbb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:01.804114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee086f-ae5e-43f7-952f-bf8ce693bbb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:01.808208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee086f-ae5e-43f7-952f-bf8ce693bbb1 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:01.951073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1285/2529] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:36:01 2022] POST /identity/v3/projects => generated 373 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:36:01.957563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a7b10-d5e4-43c3-87b8-a23ceebfefd8 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:01.977697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a7b10-d5e4-43c3-87b8-a23ceebfefd8 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:02.054859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:02.070460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e42c8-091d-4711-bd32-5dc968a50153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:02.071157 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e42c8-091d-4711-bd32-5dc968a50153 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:02.071450 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e42c8-091d-4711-bd32-5dc968a50153 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:02.071759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e42c8-091d-4711-bd32-5dc968a50153 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:02.072440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e42c8-091d-4711-bd32-5dc968a50153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:02.075050 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e42c8-091d-4711-bd32-5dc968a50153 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:02.178346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1245/2530] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:36:01 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:36:02.184187 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee086f-ae5e-43f7-952f-bf8ce693bbb1 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:02.195815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee086f-ae5e-43f7-952f-bf8ce693bbb1 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:02.200864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb770171-eaad-413f-9133-9e19b154e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:02.201619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb770171-eaad-413f-9133-9e19b154e879 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:02.201800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb770171-eaad-413f-9133-9e19b154e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:02.202071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb770171-eaad-413f-9133-9e19b154e879 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:02.202765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb770171-eaad-413f-9133-9e19b154e879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:02.204427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb770171-eaad-413f-9133-9e19b154e879 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:02.217632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1286/2531] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:36:02.222387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e42c8-091d-4711-bd32-5dc968a50153 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:02.234811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e42c8-091d-4711-bd32-5dc968a50153 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:02.240661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5992ea3-bad3-432e-8e5d-a233325fa5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:02.241390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5992ea3-bad3-432e-8e5d-a233325fa5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:02.241666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5992ea3-bad3-432e-8e5d-a233325fa5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:02.241943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5992ea3-bad3-432e-8e5d-a233325fa5e3 admin admin] PATH_INFO: `/v3/projects/8a19213ca8c34c428ea890bbcd5fea6c/users/b020a65b358b45d288bda77bde1897be/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:02.260687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5992ea3-bad3-432e-8e5d-a233325fa5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a19213ca8c34c428ea890bbcd5fea6c, user_id=b020a65b358b45d288bda77bde1897be, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:02.264239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5992ea3-bad3-432e-8e5d-a233325fa5e3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:02.341346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1246/2532] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:36:02 2022] PUT /identity/v3/projects/8a19213ca8c34c428ea890bbcd5fea6c/users/b020a65b358b45d288bda77bde1897be/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:02.345984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb770171-eaad-413f-9133-9e19b154e879 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:02.358253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb770171-eaad-413f-9133-9e19b154e879 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:02.393891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:02.441462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeee8bb-d1cc-4512-b57f-7f7bbac15f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:02.442169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee8bb-d1cc-4512-b57f-7f7bbac15f4c admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:02.442518 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee8bb-d1cc-4512-b57f-7f7bbac15f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:02.443167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee8bb-d1cc-4512-b57f-7f7bbac15f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:02.443848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeee8bb-d1cc-4512-b57f-7f7bbac15f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:02.445506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeee8bb-d1cc-4512-b57f-7f7bbac15f4c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:02.457801 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1287/2533] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:36:02.464671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5992ea3-bad3-432e-8e5d-a233325fa5e3 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:02.477815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5992ea3-bad3-432e-8e5d-a233325fa5e3 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:02.482918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965068ea-879b-4a98-bca4-e517a0fcc06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:02.483691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965068ea-879b-4a98-bca4-e517a0fcc06a admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:02.484015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965068ea-879b-4a98-bca4-e517a0fcc06a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:02.484329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965068ea-879b-4a98-bca4-e517a0fcc06a admin admin] PATH_INFO: `/v3/projects/8a19213ca8c34c428ea890bbcd5fea6c/users/b020a65b358b45d288bda77bde1897be/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:02.486337 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965068ea-879b-4a98-bca4-e517a0fcc06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a19213ca8c34c428ea890bbcd5fea6c, user_id=b020a65b358b45d288bda77bde1897be, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:02.490077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965068ea-879b-4a98-bca4-e517a0fcc06a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:02.507064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1247/2534] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:36:02 2022] PUT /identity/v3/projects/8a19213ca8c34c428ea890bbcd5fea6c/users/b020a65b358b45d288bda77bde1897be/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:36:02.518651 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4501b-78c0-401c-bbc5-39168ca8eb44 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:02.519067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4501b-78c0-401c-bbc5-39168ca8eb44 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:02.519468 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4501b-78c0-401c-bbc5-39168ca8eb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:02.554469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-71e4501b-78c0-401c-bbc5-39168ca8eb44 None None] MFA Rules not processed for user `b020a65b358b45d288bda77bde1897be`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:02.555843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-71e4501b-78c0-401c-bbc5-39168ca8eb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:02.644571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1288/2535] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:02.657587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965068ea-879b-4a98-bca4-e517a0fcc06a admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:02.659479 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-965068ea-879b-4a98-bca4-e517a0fcc06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:02.730857 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965068ea-879b-4a98-bca4-e517a0fcc06a admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:02.734690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a7917-6b60-4a99-b914-e229ce865be5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:02.735325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a7917-6b60-4a99-b914-e229ce865be5 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:02.735601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a7917-6b60-4a99-b914-e229ce865be5 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:02.735866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a7917-6b60-4a99-b914-e229ce865be5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:02.804012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a7917-6b60-4a99-b914-e229ce865be5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:02.805506 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a7917-6b60-4a99-b914-e229ce865be5 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:02.817808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1248/2536] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:36:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:36:07.263764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4501b-78c0-401c-bbc5-39168ca8eb44 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:07.341328 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4501b-78c0-401c-bbc5-39168ca8eb44 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:07.346070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:07.346766 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:07.347043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:07.347320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] PATH_INFO: `/v3/users/95439da9d9c545d5af8624feaf9dffd4` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:07.348685 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] RBAC: Authorizing `identity:delete_user(user_id=95439da9d9c545d5af8624feaf9dffd4)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:07.350857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:07.631124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95439da9d9c545d5af8624feaf9dffd4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:07.632783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95439da9d9c545d5af8624feaf9dffd4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:07.633199 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95439da9d9c545d5af8624feaf9dffd4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:07.754354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95439da9d9c545d5af8624feaf9dffd4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:07.757304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:07.764718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95439da9d9c545d5af8624feaf9dffd4'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:07.784940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1289/2537] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:36:07 2022] DELETE /identity/v3/users/95439da9d9c545d5af8624feaf9dffd4 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:07.790443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694acda2-7174-413c-a35d-c7479947f7eb None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:07.791069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694acda2-7174-413c-a35d-c7479947f7eb None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:07.791069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694acda2-7174-413c-a35d-c7479947f7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:07.813681 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-694acda2-7174-413c-a35d-c7479947f7eb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:07.815096 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-694acda2-7174-413c-a35d-c7479947f7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:07.879906 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:07.893272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1249/2538] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:07.905230 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:07.907032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:07.979113 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6185b72-50c8-4bbb-8c83-85a31b7d422b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:07.983129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a99cb-30ff-48ba-a1f7-d3e6e438449e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:07.983764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a99cb-30ff-48ba-a1f7-d3e6e438449e service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:07.984043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a99cb-30ff-48ba-a1f7-d3e6e438449e service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:07.984316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a99cb-30ff-48ba-a1f7-d3e6e438449e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:07.994794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a99cb-30ff-48ba-a1f7-d3e6e438449e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:07.996264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a99cb-30ff-48ba-a1f7-d3e6e438449e service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:08.008809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1290/2539] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:36:08.543972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694acda2-7174-413c-a35d-c7479947f7eb None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:08.543972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-694acda2-7174-413c-a35d-c7479947f7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:08.562731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694acda2-7174-413c-a35d-c7479947f7eb None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:08.571486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:08.572202 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:08.572486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:08.572796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] PATH_INFO: `/v3/projects/8d345a815fcf4d4bb57d98c8dbd60a09` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:08.574031 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d345a815fcf4d4bb57d98c8dbd60a09)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:08.576017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:08.710602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d345a815fcf4d4bb57d98c8dbd60a09', 'request_id': 'req-f9f91258-a932-4545-8e29-424335e98197'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:08.718117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d345a815fcf4d4bb57d98c8dbd60a09', 'request_id': 'req-f9f91258-a932-4545-8e29-424335e98197'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:08.814369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d345a815fcf4d4bb57d98c8dbd60a09', 'request_id': 'req-f9f91258-a932-4545-8e29-424335e98197'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:08.829350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] The token cache is being invalidate because project 8d345a815fcf4d4bb57d98c8dbd60a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:36:08.829776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:08.830659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:08.836245 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1250/2540] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:36:08 2022] DELETE /identity/v3/projects/8d345a815fcf4d4bb57d98c8dbd60a09 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:36:08.837476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a99cb-30ff-48ba-a1f7-d3e6e438449e service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:08.849311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a99cb-30ff-48ba-a1f7-d3e6e438449e service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:08.914903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:08.915562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:08.915841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:08.916112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] PATH_INFO: `/v3/users/b020a65b358b45d288bda77bde1897be` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:08.917346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b020a65b358b45d288bda77bde1897be)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:08.919539 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:09.149017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b020a65b358b45d288bda77bde1897be'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:09.149897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b020a65b358b45d288bda77bde1897be'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:09.150874 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b020a65b358b45d288bda77bde1897be'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:09.257616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b020a65b358b45d288bda77bde1897be'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:09.266279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8d7bfce6-0560-4e61-a5ae-ade740cc0829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b020a65b358b45d288bda77bde1897be'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:09.279632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:09.289045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1291/2541] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:36:08 2022] DELETE /identity/v3/users/b020a65b358b45d288bda77bde1897be => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:09.804943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:09.804943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:09.824974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f91258-a932-4545-8e29-424335e98197 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:09.908415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:09.909257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:09.909639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:09.910034 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] PATH_INFO: `/v3/projects/8a19213ca8c34c428ea890bbcd5fea6c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:09.912316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a19213ca8c34c428ea890bbcd5fea6c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:09.914330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:10.100499 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a19213ca8c34c428ea890bbcd5fea6c', 'request_id': 'req-fc2d4277-fb99-4208-a80e-c7fdfc9391de'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:10.107623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a19213ca8c34c428ea890bbcd5fea6c', 'request_id': 'req-fc2d4277-fb99-4208-a80e-c7fdfc9391de'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:10.199779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a19213ca8c34c428ea890bbcd5fea6c', 'request_id': 'req-fc2d4277-fb99-4208-a80e-c7fdfc9391de'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:10.214579 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] The token cache is being invalidate because project 8a19213ca8c34c428ea890bbcd5fea6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:36:10.219371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:10.219371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fc2d4277-fb99-4208-a80e-c7fdfc9391de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:10.219371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1251/2542] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:36:09 2022] DELETE /identity/v3/projects/8a19213ca8c34c428ea890bbcd5fea6c => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:10.223799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677a50b-2101-47a6-810c-ded35c0d50d1 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:10.225311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677a50b-2101-47a6-810c-ded35c0d50d1 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:10.225311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677a50b-2101-47a6-810c-ded35c0d50d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:10.246955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8677a50b-2101-47a6-810c-ded35c0d50d1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:10.248238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8677a50b-2101-47a6-810c-ded35c0d50d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:10.321042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1292/2543] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:10.330768 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2010e8-c673-4fb6-aa7d-e67fe3d9758b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:10.331036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2010e8-c673-4fb6-aa7d-e67fe3d9758b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:10.331315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2010e8-c673-4fb6-aa7d-e67fe3d9758b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:10.349408 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:10.355830 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ea2010e8-c673-4fb6-aa7d-e67fe3d9758b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:10.356971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ea2010e8-c673-4fb6-aa7d-e67fe3d9758b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:10.365607 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1252/2544] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:10.370038 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677a50b-2101-47a6-810c-ded35c0d50d1 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:10.380829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677a50b-2101-47a6-810c-ded35c0d50d1 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:10.386589 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58925a5-0e0e-42ed-a590-f14cbaad09ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:10.387227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58925a5-0e0e-42ed-a590-f14cbaad09ec admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:10.387505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58925a5-0e0e-42ed-a590-f14cbaad09ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:10.387778 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58925a5-0e0e-42ed-a590-f14cbaad09ec admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:10.388448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58925a5-0e0e-42ed-a590-f14cbaad09ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:36:10.388754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58925a5-0e0e-42ed-a590-f14cbaad09ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:10.390432 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58925a5-0e0e-42ed-a590-f14cbaad09ec admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:10.404643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1293/2545] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:36:10.413168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2010e8-c673-4fb6-aa7d-e67fe3d9758b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:10.430961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2010e8-c673-4fb6-aa7d-e67fe3d9758b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:10.450244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa2633-62b8-4b84-af10-911d1a3d8aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:10.450244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa2633-62b8-4b84-af10-911d1a3d8aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:10.450244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa2633-62b8-4b84-af10-911d1a3d8aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:10.450244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa2633-62b8-4b84-af10-911d1a3d8aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:10.450836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa2633-62b8-4b84-af10-911d1a3d8aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:10.450836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa2633-62b8-4b84-af10-911d1a3d8aa2 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:10.640903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1253/2546] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:36:10 2022] POST /identity/v3/projects => generated 373 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:36:10.645837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58925a5-0e0e-42ed-a590-f14cbaad09ec admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:10.657394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58925a5-0e0e-42ed-a590-f14cbaad09ec admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:10.735884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bde636-2eaa-4ffb-9c0e-ef68f1abf848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:10.736512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bde636-2eaa-4ffb-9c0e-ef68f1abf848 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:10.736787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bde636-2eaa-4ffb-9c0e-ef68f1abf848 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:10.737057 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bde636-2eaa-4ffb-9c0e-ef68f1abf848 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:10.737687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bde636-2eaa-4ffb-9c0e-ef68f1abf848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:10.740922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bde636-2eaa-4ffb-9c0e-ef68f1abf848 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:10.847105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1294/2547] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:36:10 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:36:10.851610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa2633-62b8-4b84-af10-911d1a3d8aa2 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:10.863259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa2633-62b8-4b84-af10-911d1a3d8aa2 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:10.868860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2378bb-70dc-427c-a568-fab2cd03f603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:10.869483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2378bb-70dc-427c-a568-fab2cd03f603 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:10.869763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2378bb-70dc-427c-a568-fab2cd03f603 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:10.870054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2378bb-70dc-427c-a568-fab2cd03f603 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:10.870742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2378bb-70dc-427c-a568-fab2cd03f603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:10.872625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2378bb-70dc-427c-a568-fab2cd03f603 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:10.889509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1254/2548] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:36:10.892896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bde636-2eaa-4ffb-9c0e-ef68f1abf848 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:10.908385 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bde636-2eaa-4ffb-9c0e-ef68f1abf848 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:10.913682 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a46089-4e41-4c8a-9efa-86750b64475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:10.925537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a46089-4e41-4c8a-9efa-86750b64475a admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:10.925537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a46089-4e41-4c8a-9efa-86750b64475a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:10.925537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a46089-4e41-4c8a-9efa-86750b64475a admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:10.925537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a46089-4e41-4c8a-9efa-86750b64475a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:10.925537 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a46089-4e41-4c8a-9efa-86750b64475a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:10.929571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1295/2549] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:36:10.933851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2378bb-70dc-427c-a568-fab2cd03f603 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:10.945934 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2378bb-70dc-427c-a568-fab2cd03f603 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:10.951309 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda9fc6-3915-431f-8033-0dba2e501f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:10.952012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda9fc6-3915-431f-8033-0dba2e501f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:10.952293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda9fc6-3915-431f-8033-0dba2e501f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:10.952599 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda9fc6-3915-431f-8033-0dba2e501f04 admin admin] PATH_INFO: `/v3/projects/608e3c25b97f4a75b2a80903a9d5c656/users/9454a7a736164ff09382babab30694d8/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:10.971499 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda9fc6-3915-431f-8033-0dba2e501f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=608e3c25b97f4a75b2a80903a9d5c656, user_id=9454a7a736164ff09382babab30694d8, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:10.974806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda9fc6-3915-431f-8033-0dba2e501f04 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:11.071287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1255/2550] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:36:10 2022] PUT /identity/v3/projects/608e3c25b97f4a75b2a80903a9d5c656/users/9454a7a736164ff09382babab30694d8/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:11.077084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2434-5c05-4f7e-999e-c57448f584dc None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:11.077513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2434-5c05-4f7e-999e-c57448f584dc None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:11.077930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2434-5c05-4f7e-999e-c57448f584dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:11.102008 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e1bc2434-5c05-4f7e-999e-c57448f584dc None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:11.103311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e1bc2434-5c05-4f7e-999e-c57448f584dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:11.168475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:11.179615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1296/2551] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:11.191330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda9fc6-3915-431f-8033-0dba2e501f04 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:11.194697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-dbda9fc6-3915-431f-8033-0dba2e501f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:11.288550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda9fc6-3915-431f-8033-0dba2e501f04 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:11.292649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c100307-e1ff-410f-a3e2-e9f5f1130aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:11.293327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c100307-e1ff-410f-a3e2-e9f5f1130aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:11.293605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c100307-e1ff-410f-a3e2-e9f5f1130aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:11.293875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c100307-e1ff-410f-a3e2-e9f5f1130aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:11.305257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c100307-e1ff-410f-a3e2-e9f5f1130aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:11.306856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c100307-e1ff-410f-a3e2-e9f5f1130aa4 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:11.321112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1256/2552] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:36:19.120016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fac5b-4540-4252-b467-f71dfaaf8775 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:19.120802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fac5b-4540-4252-b467-f71dfaaf8775 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:19.120802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fac5b-4540-4252-b467-f71dfaaf8775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:19.159864 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-3d9fac5b-4540-4252-b467-f71dfaaf8775 None None] MFA Rules not processed for user `9454a7a736164ff09382babab30694d8`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:19.161880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-3d9fac5b-4540-4252-b467-f71dfaaf8775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:19.280173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1297/2553] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:36:19.287168 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c100307-e1ff-410f-a3e2-e9f5f1130aa4 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:19.299092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c100307-e1ff-410f-a3e2-e9f5f1130aa4 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:19.304993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90533f-8158-4142-b28e-3eb783b0dbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:19.305625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90533f-8158-4142-b28e-3eb783b0dbf9 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:19.305909 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90533f-8158-4142-b28e-3eb783b0dbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:19.306149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90533f-8158-4142-b28e-3eb783b0dbf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:19.306858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90533f-8158-4142-b28e-3eb783b0dbf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:19.308962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90533f-8158-4142-b28e-3eb783b0dbf9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:19.417095 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1257/2554] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:36:19 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:36:19.422085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fac5b-4540-4252-b467-f71dfaaf8775 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:19.440164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fac5b-4540-4252-b467-f71dfaaf8775 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:19.555876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fbec3-bc70-4855-bffd-90d5ef2f9388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:19.556838 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fbec3-bc70-4855-bffd-90d5ef2f9388 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:19.557322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fbec3-bc70-4855-bffd-90d5ef2f9388 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:19.557816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fbec3-bc70-4855-bffd-90d5ef2f9388 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:19.558994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fbec3-bc70-4855-bffd-90d5ef2f9388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:19.562918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fbec3-bc70-4855-bffd-90d5ef2f9388 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:19.657288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1298/2555] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:36:19 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:36:19.663087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90533f-8158-4142-b28e-3eb783b0dbf9 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:19.674350 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90533f-8158-4142-b28e-3eb783b0dbf9 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:19.679588 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d55af-6ddf-4dfa-8dbc-dfa6ef79099b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:19.680212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d55af-6ddf-4dfa-8dbc-dfa6ef79099b admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:19.680488 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d55af-6ddf-4dfa-8dbc-dfa6ef79099b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:19.680763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d55af-6ddf-4dfa-8dbc-dfa6ef79099b admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:19.681405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d55af-6ddf-4dfa-8dbc-dfa6ef79099b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:19.683082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d55af-6ddf-4dfa-8dbc-dfa6ef79099b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:19.699163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1258/2556] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:36:19.705861 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fbec3-bc70-4855-bffd-90d5ef2f9388 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:19.727028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fbec3-bc70-4855-bffd-90d5ef2f9388 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:19.734982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f0ae2-fc3f-4940-a20b-f3b9c8b4e041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:19.736121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f0ae2-fc3f-4940-a20b-f3b9c8b4e041 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:19.736609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f0ae2-fc3f-4940-a20b-f3b9c8b4e041 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:19.737226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f0ae2-fc3f-4940-a20b-f3b9c8b4e041 admin admin] PATH_INFO: `/v3/projects/4f5b48501d8c4d55a151a57d1d271d9d/users/f26628e45f234578a7d582bca5aede59/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:19.769301 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f0ae2-fc3f-4940-a20b-f3b9c8b4e041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5b48501d8c4d55a151a57d1d271d9d, user_id=f26628e45f234578a7d582bca5aede59, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:19.774127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f0ae2-fc3f-4940-a20b-f3b9c8b4e041 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:19.855666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1299/2557] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:36:19 2022] PUT /identity/v3/projects/4f5b48501d8c4d55a151a57d1d271d9d/users/f26628e45f234578a7d582bca5aede59/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:19.858869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d55af-6ddf-4dfa-8dbc-dfa6ef79099b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:19.871994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d55af-6ddf-4dfa-8dbc-dfa6ef79099b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:19.996030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57165dda-83f9-457a-9d4c-4e9d450ae9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:19.996824 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57165dda-83f9-457a-9d4c-4e9d450ae9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:19.997301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57165dda-83f9-457a-9d4c-4e9d450ae9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:19.997816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57165dda-83f9-457a-9d4c-4e9d450ae9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:19.999698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57165dda-83f9-457a-9d4c-4e9d450ae9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:20.001391 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57165dda-83f9-457a-9d4c-4e9d450ae9b8 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:20.018587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1259/2558] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:36:20.020643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f0ae2-fc3f-4940-a20b-f3b9c8b4e041 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:20.037046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f0ae2-fc3f-4940-a20b-f3b9c8b4e041 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:20.042670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8485dd-b3bb-4da0-a157-690ccec7f0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:20.043918 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8485dd-b3bb-4da0-a157-690ccec7f0ef admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:20.044188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8485dd-b3bb-4da0-a157-690ccec7f0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:20.044442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8485dd-b3bb-4da0-a157-690ccec7f0ef admin admin] PATH_INFO: `/v3/projects/4f5b48501d8c4d55a151a57d1d271d9d/users/f26628e45f234578a7d582bca5aede59/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:20.047681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8485dd-b3bb-4da0-a157-690ccec7f0ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5b48501d8c4d55a151a57d1d271d9d, user_id=f26628e45f234578a7d582bca5aede59, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:20.052144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8485dd-b3bb-4da0-a157-690ccec7f0ef admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:20.068906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1300/2559] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:36:20 2022] PUT /identity/v3/projects/4f5b48501d8c4d55a151a57d1d271d9d/users/f26628e45f234578a7d582bca5aede59/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:36:28.997605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596864-da85-4fcc-b818-753cb6954099 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:28.998280 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596864-da85-4fcc-b818-753cb6954099 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:28.998605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596864-da85-4fcc-b818-753cb6954099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:29.027571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:29.045012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-70596864-da85-4fcc-b818-753cb6954099 None None] MFA Rules not processed for user `f26628e45f234578a7d582bca5aede59`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:29.047250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-70596864-da85-4fcc-b818-753cb6954099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:29.151561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1260/2560] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:29.167628 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8485dd-b3bb-4da0-a157-690ccec7f0ef admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:29.170630 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6e8485dd-b3bb-4da0-a157-690ccec7f0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:29.239354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:29.288383 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8485dd-b3bb-4da0-a157-690ccec7f0ef admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:29.295476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4a3a9-bff2-4e2f-bb3e-9ebafb47ad0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:29.296486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4a3a9-bff2-4e2f-bb3e-9ebafb47ad0f service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:29.296945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4a3a9-bff2-4e2f-bb3e-9ebafb47ad0f service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:29.297348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4a3a9-bff2-4e2f-bb3e-9ebafb47ad0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:29.411255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4a3a9-bff2-4e2f-bb3e-9ebafb47ad0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:29.414625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4a3a9-bff2-4e2f-bb3e-9ebafb47ad0f service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:29.453074 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1301/2561] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:36:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:36:30.327923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70596864-da85-4fcc-b818-753cb6954099 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:30.331282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-70596864-da85-4fcc-b818-753cb6954099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:30.456164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70596864-da85-4fcc-b818-753cb6954099 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:30.465608 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbf93f-36ce-4879-8808-1bdcef97d1fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:30.467228 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbf93f-36ce-4879-8808-1bdcef97d1fd service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:30.467903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbf93f-36ce-4879-8808-1bdcef97d1fd service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:30.468535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbf93f-36ce-4879-8808-1bdcef97d1fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:30.472828 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-72cbf93f-36ce-4879-8808-1bdcef97d1fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:30.538779 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbf93f-36ce-4879-8808-1bdcef97d1fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:30.541894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbf93f-36ce-4879-8808-1bdcef97d1fd service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:30.574447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1261/2562] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:36:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:36:33.766625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4a3a9-bff2-4e2f-bb3e-9ebafb47ad0f service nova] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:33.771114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f9c4a3a9-bff2-4e2f-bb3e-9ebafb47ad0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:33.800170 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4a3a9-bff2-4e2f-bb3e-9ebafb47ad0f service nova] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:33.807263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c07a7-465e-4271-a94f-778817dc6bdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:33.809034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c07a7-465e-4271-a94f-778817dc6bdb service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:33.809904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c07a7-465e-4271-a94f-778817dc6bdb service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:33.810788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c07a7-465e-4271-a94f-778817dc6bdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:33.814899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b66c07a7-465e-4271-a94f-778817dc6bdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:33.850020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c07a7-465e-4271-a94f-778817dc6bdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:33.852210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c07a7-465e-4271-a94f-778817dc6bdb service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:33.866812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1302/2563] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:36:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:36:34.292741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15633cd3-3d22-44e1-b9d5-ea6722e6e2c9 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:34.293601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15633cd3-3d22-44e1-b9d5-ea6722e6e2c9 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:34.293601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15633cd3-3d22-44e1-b9d5-ea6722e6e2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:34.318807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-15633cd3-3d22-44e1-b9d5-ea6722e6e2c9 None None] MFA Rules not processed for user `9454a7a736164ff09382babab30694d8`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:34.320255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-15633cd3-3d22-44e1-b9d5-ea6722e6e2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:34.338257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1262/2564] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:36.116784 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c07a7-465e-4271-a94f-778817dc6bdb service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:36.120208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b66c07a7-465e-4271-a94f-778817dc6bdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:36.147849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c07a7-465e-4271-a94f-778817dc6bdb service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:36.175714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68d393-dbd7-4fa4-b988-f9581ceb0034 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:36.177536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68d393-dbd7-4fa4-b988-f9581ceb0034 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:36.178096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68d393-dbd7-4fa4-b988-f9581ceb0034 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:36.178673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68d393-dbd7-4fa4-b988-f9581ceb0034 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:36.180924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0a68d393-dbd7-4fa4-b988-f9581ceb0034 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:36.376554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68d393-dbd7-4fa4-b988-f9581ceb0034 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:36.379151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68d393-dbd7-4fa4-b988-f9581ceb0034 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:36.415209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1303/2565] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:36:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:36:41.080920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b81fd-270d-49fb-ae47-b0706f0da098 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:41.083199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b81fd-270d-49fb-ae47-b0706f0da098 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:41.083199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b81fd-270d-49fb-ae47-b0706f0da098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:41.120126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-db1b81fd-270d-49fb-ae47-b0706f0da098 None None] MFA Rules not processed for user `b0db362075b04b2294a6268576a5bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:41.121424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-db1b81fd-270d-49fb-ae47-b0706f0da098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:41.130334 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1263/2566] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:36:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:36:41.204104 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d552e2-cf7c-4433-80d1-4b2b4f5b18eb None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:41.204629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d552e2-cf7c-4433-80d1-4b2b4f5b18eb None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:41.204800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d552e2-cf7c-4433-80d1-4b2b4f5b18eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:41.229442 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-08d552e2-cf7c-4433-80d1-4b2b4f5b18eb None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:36:41.230807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-08d552e2-cf7c-4433-80d1-4b2b4f5b18eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:41.269368 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:41.305249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1304/2567] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 22 17:36:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:36:41.478382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b81fd-270d-49fb-ae47-b0706f0da098 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:41.480056 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-db1b81fd-270d-49fb-ae47-b0706f0da098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:41.605940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b81fd-270d-49fb-ae47-b0706f0da098 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:41.611223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c4261-399c-4d2b-955f-1c4c99bef2d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:41.611854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c4261-399c-4d2b-955f-1c4c99bef2d2 service swift] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:41.612136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c4261-399c-4d2b-955f-1c4c99bef2d2 service swift] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:41.612419 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c4261-399c-4d2b-955f-1c4c99bef2d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:41.623859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c4261-399c-4d2b-955f-1c4c99bef2d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:41.626860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c4261-399c-4d2b-955f-1c4c99bef2d2 service swift] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:41.637778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1264/2568] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:36:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:36:44.841652 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d552e2-cf7c-4433-80d1-4b2b4f5b18eb None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:44.904995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d552e2-cf7c-4433-80d1-4b2b4f5b18eb None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:44.909584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf87fe-4416-4993-9722-f758c6074304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:44.910391 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf87fe-4416-4993-9722-f758c6074304 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:44.913759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf87fe-4416-4993-9722-f758c6074304 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:44.913759 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf87fe-4416-4993-9722-f758c6074304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:44.918045 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c0cf87fe-4416-4993-9722-f758c6074304 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:45.090929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf87fe-4416-4993-9722-f758c6074304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:45.093939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf87fe-4416-4993-9722-f758c6074304 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:45.117880 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1305/2569] 10.222.0.20 () {60 vars in 1396 bytes} [Mon Aug 22 17:36:44 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:36:45.807985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c4261-399c-4d2b-955f-1c4c99bef2d2 service swift] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:45.824326 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c4261-399c-4d2b-955f-1c4c99bef2d2 service swift] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:45.829452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c2b99-6789-41bb-8d9d-4e8084e2840c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:45.830340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c2b99-6789-41bb-8d9d-4e8084e2840c service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:45.830798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c2b99-6789-41bb-8d9d-4e8084e2840c service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:45.831915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c2b99-6789-41bb-8d9d-4e8084e2840c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:45.835072 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-c30c2b99-6789-41bb-8d9d-4e8084e2840c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:45.842566 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:45.967197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c2b99-6789-41bb-8d9d-4e8084e2840c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:45.970417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c2b99-6789-41bb-8d9d-4e8084e2840c service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:45.992135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1265/2570] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:36:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:36:55.110678 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=66 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:55.328612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf87fe-4416-4993-9722-f758c6074304 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:55.328612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c0cf87fe-4416-4993-9722-f758c6074304 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:55.344797 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf87fe-4416-4993-9722-f758c6074304 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:55.351394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:55.352061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:55.352347 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:55.352653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] PATH_INFO: `/v3/users/aff4cd6b5ec14cb29813416b1d929333` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:55.353869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff4cd6b5ec14cb29813416b1d929333)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:55.355905 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:55.600154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff4cd6b5ec14cb29813416b1d929333'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:55.600901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff4cd6b5ec14cb29813416b1d929333'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:55.601538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff4cd6b5ec14cb29813416b1d929333'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:55.667072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff4cd6b5ec14cb29813416b1d929333'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:55.676837 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff4cd6b5ec14cb29813416b1d929333'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:55.697137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1306/2571] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:36:55 2022] DELETE /identity/v3/users/aff4cd6b5ec14cb29813416b1d929333 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:56.102338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=27 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:56.200462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c2b99-6789-41bb-8d9d-4e8084e2840c service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:56.219648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c2b99-6789-41bb-8d9d-4e8084e2840c service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:56.302648 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:56.303727 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:56.304220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:56.304719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] PATH_INFO: `/v3/projects/11871f979145455499b30912da1abcfd` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:56.306802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] RBAC: Authorizing `identity:delete_project(project_id=11871f979145455499b30912da1abcfd)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:56.310599 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:56.520305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11871f979145455499b30912da1abcfd', 'request_id': 'req-6a429e28-0443-4867-9d48-5672cd51edea'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:56.528353 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11871f979145455499b30912da1abcfd', 'request_id': 'req-6a429e28-0443-4867-9d48-5672cd51edea'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:56.642448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11871f979145455499b30912da1abcfd', 'request_id': 'req-6a429e28-0443-4867-9d48-5672cd51edea'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:56.663498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] The token cache is being invalidate because project 11871f979145455499b30912da1abcfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:36:56.664001 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:56.665546 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:56.668149 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1266/2572] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:36:56 2022] DELETE /identity/v3/projects/11871f979145455499b30912da1abcfd => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:56.674333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:56.676709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:56.698979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3d0a5-8f34-4160-9b19-d6b3a6a0d185 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:56.788449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:56.789119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:56.789398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:56.789681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] PATH_INFO: `/v3/users/192fbd27593e434591a091944ccfcfc5` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:56.791014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=192fbd27593e434591a091944ccfcfc5)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:56.792994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:57.031667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192fbd27593e434591a091944ccfcfc5'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:57.032556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192fbd27593e434591a091944ccfcfc5'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:57.033335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192fbd27593e434591a091944ccfcfc5'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:57.173059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192fbd27593e434591a091944ccfcfc5'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:57.185649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7d0b1a7-4a8b-48ba-8cc3-96f1a9456d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192fbd27593e434591a091944ccfcfc5'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:57.201349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:57.223378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1307/2573] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:36:56 2022] DELETE /identity/v3/users/192fbd27593e434591a091944ccfcfc5 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:36:57.551458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:36:57.553126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:36:57.567397 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a429e28-0443-4867-9d48-5672cd51edea admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:36:57.662471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:36:57.663720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:36:57.664090 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:36:57.664423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] PATH_INFO: `/v3/projects/d275a9099ec54100956f75274cd16bbb` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:36:57.666150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d275a9099ec54100956f75274cd16bbb)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:36:57.668610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:36:57.777500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:36:57.800093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd275a9099ec54100956f75274cd16bbb', 'request_id': 'req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:57.812349 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd275a9099ec54100956f75274cd16bbb', 'request_id': 'req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:57.933462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd275a9099ec54100956f75274cd16bbb', 'request_id': 'req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:57.947517 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] The token cache is being invalidate because project d275a9099ec54100956f75274cd16bbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:36:57.947950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:57.948792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:36:57.950738 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1267/2574] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:36:57 2022] DELETE /identity/v3/projects/d275a9099ec54100956f75274cd16bbb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:37:05.364544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093378f3-a3c2-4b2f-b5e0-245fee2eed10 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:05.365154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093378f3-a3c2-4b2f-b5e0-245fee2eed10 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:05.365585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093378f3-a3c2-4b2f-b5e0-245fee2eed10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:05.435289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-093378f3-a3c2-4b2f-b5e0-245fee2eed10 None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:37:05.436926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-093378f3-a3c2-4b2f-b5e0-245fee2eed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:05.525525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1308/2575] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:37:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:37:05.542698 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:37:05.544473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:05.636591 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4abdf-ad9c-42e1-8997-5d8b2d624abb admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:37:05.644402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499738f-dda8-41fe-a273-a1d7d695e6ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:37:05.645232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499738f-dda8-41fe-a273-a1d7d695e6ab service swift] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:05.645704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499738f-dda8-41fe-a273-a1d7d695e6ab service swift] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:05.646188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499738f-dda8-41fe-a273-a1d7d695e6ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:05.657649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499738f-dda8-41fe-a273-a1d7d695e6ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:37:05.659861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499738f-dda8-41fe-a273-a1d7d695e6ab service swift] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:37:05.671038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1268/2576] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:37:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:37:15.521585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=43,keystone:DELETE=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:37:15.666437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=44,keystone:DELETE=5,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:37:19.898512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093378f3-a3c2-4b2f-b5e0-245fee2eed10 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:37:19.900273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-093378f3-a3c2-4b2f-b5e0-245fee2eed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:19.968357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093378f3-a3c2-4b2f-b5e0-245fee2eed10 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:37:19.973378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf479b-6c3e-4320-9bb6-da82391f2242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:37:19.974042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf479b-6c3e-4320-9bb6-da82391f2242 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:19.974420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf479b-6c3e-4320-9bb6-da82391f2242 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:19.974781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf479b-6c3e-4320-9bb6-da82391f2242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:19.977294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-27cf479b-6c3e-4320-9bb6-da82391f2242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:19.988556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf479b-6c3e-4320-9bb6-da82391f2242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:37:19.990342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf479b-6c3e-4320-9bb6-da82391f2242 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:37:19.999427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1309/2577] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:37:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:37:25.851967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499738f-dda8-41fe-a273-a1d7d695e6ab service swift] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:37:25.863964 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499738f-dda8-41fe-a273-a1d7d695e6ab service swift] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:37:25.870815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1253f-c2f5-453f-b5cd-3080bf02137c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:37:25.870815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1253f-c2f5-453f-b5cd-3080bf02137c service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:25.870815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1253f-c2f5-453f-b5cd-3080bf02137c service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:25.870815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1253f-c2f5-453f-b5cd-3080bf02137c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:25.872094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5de1253f-c2f5-453f-b5cd-3080bf02137c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:25.957209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1253f-c2f5-453f-b5cd-3080bf02137c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:37:25.958754 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1253f-c2f5-453f-b5cd-3080bf02137c service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:37:25.971209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1269/2578] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:37:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:37:26.141388 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf479b-6c3e-4320-9bb6-da82391f2242 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:37:26.143561 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-27cf479b-6c3e-4320-9bb6-da82391f2242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:26.259883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf479b-6c3e-4320-9bb6-da82391f2242 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:37:26.267342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ddbf3-c742-4a3d-b466-85232aa0938e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea4c2870e2754f3188228fc441950df2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:37:26.268239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ddbf3-c742-4a3d-b466-85232aa0938e service glance] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:26.268724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ddbf3-c742-4a3d-b466-85232aa0938e service glance] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:26.269085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ddbf3-c742-4a3d-b466-85232aa0938e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:26.275743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-347ddbf3-c742-4a3d-b466-85232aa0938e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:26.435660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ddbf3-c742-4a3d-b466-85232aa0938e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:37:26.437897 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ddbf3-c742-4a3d-b466-85232aa0938e service glance] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:37:26.453626 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1310/2579] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:37:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:37:26.687476 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1253f-c2f5-453f-b5cd-3080bf02137c service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:37:26.697331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1253f-c2f5-453f-b5cd-3080bf02137c service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:37:26.701338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48c472-6d9e-44f8-a816-328f24942952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:37:26.701965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48c472-6d9e-44f8-a816-328f24942952 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:26.702239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48c472-6d9e-44f8-a816-328f24942952 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:26.702654 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48c472-6d9e-44f8-a816-328f24942952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:26.704905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1b48c472-6d9e-44f8-a816-328f24942952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:26.717552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b48c472-6d9e-44f8-a816-328f24942952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:37:26.721313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b48c472-6d9e-44f8-a816-328f24942952 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:37:26.736428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1270/2580] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:37:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:37:26.831653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ddbf3-c742-4a3d-b466-85232aa0938e service glance] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:37:26.846247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ddbf3-c742-4a3d-b466-85232aa0938e service glance] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:37:26.852217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acda53-57db-4105-b666-b444b8641a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:37:26.853210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acda53-57db-4105-b666-b444b8641a3a service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:26.853705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acda53-57db-4105-b666-b444b8641a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:26.854191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acda53-57db-4105-b666-b444b8641a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:26.857336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b6acda53-57db-4105-b666-b444b8641a3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:26.884361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acda53-57db-4105-b666-b444b8641a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:37:26.887237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acda53-57db-4105-b666-b444b8641a3a service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:37:26.906773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1311/2581] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:37:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:37:27.002527 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48c472-6d9e-44f8-a816-328f24942952 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:37:27.005242 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1b48c472-6d9e-44f8-a816-328f24942952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:27.105333 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48c472-6d9e-44f8-a816-328f24942952 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:37:27.112866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9ecfd-e08e-48fa-ab3a-cfd656f1b78f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:37:27.113557 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9ecfd-e08e-48fa-ab3a-cfd656f1b78f service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:27.113835 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9ecfd-e08e-48fa-ab3a-cfd656f1b78f service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:27.114107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9ecfd-e08e-48fa-ab3a-cfd656f1b78f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:27.116151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8ae9ecfd-e08e-48fa-ab3a-cfd656f1b78f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:27.130145 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9ecfd-e08e-48fa-ab3a-cfd656f1b78f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:37:27.132005 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9ecfd-e08e-48fa-ab3a-cfd656f1b78f service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:37:27.145016 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1271/2582] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:37:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:37:28.883556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acda53-57db-4105-b666-b444b8641a3a service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:37:28.886027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b6acda53-57db-4105-b666-b444b8641a3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:28.911793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acda53-57db-4105-b666-b444b8641a3a service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:37:28.924165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce688a-d272-454e-863f-ce153b0c223a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:37:28.927547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce688a-d272-454e-863f-ce153b0c223a service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:28.927547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce688a-d272-454e-863f-ce153b0c223a service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:28.927547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce688a-d272-454e-863f-ce153b0c223a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:28.930922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-abce688a-d272-454e-863f-ce153b0c223a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:28.945850 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce688a-d272-454e-863f-ce153b0c223a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:37:28.947617 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce688a-d272-454e-863f-ce153b0c223a service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:37:28.962244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1312/2583] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:37:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:37:29.967618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deacb8-1dcf-4ff6-91f4-fd50c2603c90 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:29.968456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deacb8-1dcf-4ff6-91f4-fd50c2603c90 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:29.968456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deacb8-1dcf-4ff6-91f4-fd50c2603c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:30.002999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d7deacb8-1dcf-4ff6-91f4-fd50c2603c90 None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:37:30.002999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d7deacb8-1dcf-4ff6-91f4-fd50c2603c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:30.011169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1272/2584] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 22 17:37:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:37:30.027201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce688a-d272-454e-863f-ce153b0c223a service nova] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:37:30.055912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce688a-d272-454e-863f-ce153b0c223a service nova] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:37:30.061611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98001f3e-22e7-49bd-8735-99de0bde4a65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:37:30.062240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001f3e-22e7-49bd-8735-99de0bde4a65 service swift] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:30.062547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001f3e-22e7-49bd-8735-99de0bde4a65 service swift] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:30.062824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001f3e-22e7-49bd-8735-99de0bde4a65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:30.070026 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:37:30.072540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98001f3e-22e7-49bd-8735-99de0bde4a65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:37:30.074205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98001f3e-22e7-49bd-8735-99de0bde4a65 service swift] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:37:30.084425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1313/2585] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:37:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:37:39.986748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=71 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:37:40.105911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:37:51.173080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9264a7e-d635-417b-aad4-2e3fb0071975 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:51.173080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9264a7e-d635-417b-aad4-2e3fb0071975 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:51.173080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9264a7e-d635-417b-aad4-2e3fb0071975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:51.182599 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a9264a7e-d635-417b-aad4-2e3fb0071975 None None] MFA Rules not processed for user `9001ee719e294f79b35f13389ac9cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:37:51.184059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a9264a7e-d635-417b-aad4-2e3fb0071975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:37:51.193958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1273/2586] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 22 17:37:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:37:51.217226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98001f3e-22e7-49bd-8735-99de0bde4a65 service swift] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:37:51.228699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98001f3e-22e7-49bd-8735-99de0bde4a65 service swift] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:37:51.233971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006eadb-c833-445d-a49f-e644af989ad6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38462b1165514ed7b948cba710782d4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:37:51.234881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006eadb-c833-445d-a49f-e644af989ad6 service swift] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:37:51.235161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006eadb-c833-445d-a49f-e644af989ad6 service swift] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:37:51.235463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006eadb-c833-445d-a49f-e644af989ad6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:37:51.245779 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006eadb-c833-445d-a49f-e644af989ad6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:37:51.247891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006eadb-c833-445d-a49f-e644af989ad6 service swift] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:37:51.257453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1314/2587] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 22 17:37:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 17:38:01.177652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:38:01.254148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=6 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:38:10.950018 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323f6f0-1f5d-467f-8aae-48af356d8f6c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:38:10.950705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323f6f0-1f5d-467f-8aae-48af356d8f6c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:38:10.950705 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323f6f0-1f5d-467f-8aae-48af356d8f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:38:10.973680 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-0323f6f0-1f5d-467f-8aae-48af356d8f6c None None] MFA Rules not processed for user `aa55b9fade0047c68c2f3f6bf4dfe44e`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:38:10.975165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0323f6f0-1f5d-467f-8aae-48af356d8f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:38:11.044403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1274/2588] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:38:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:38:11.428253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006eadb-c833-445d-a49f-e644af989ad6 service swift] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:38:11.430312 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8006eadb-c833-445d-a49f-e644af989ad6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:38:11.446242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006eadb-c833-445d-a49f-e644af989ad6 service swift] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:38:11.448543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90c6d1-cf9e-4938-b124-bc9cc45ec3bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:38:11.449155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90c6d1-cf9e-4938-b124-bc9cc45ec3bf service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:38:11.449466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90c6d1-cf9e-4938-b124-bc9cc45ec3bf service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:38:11.449816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90c6d1-cf9e-4938-b124-bc9cc45ec3bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:38:11.451638 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9f90c6d1-cf9e-4938-b124-bc9cc45ec3bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:38:11.464882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90c6d1-cf9e-4938-b124-bc9cc45ec3bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:38:11.467858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90c6d1-cf9e-4938-b124-bc9cc45ec3bf service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:38:11.481852 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1315/2589] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:38:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:38:21.036931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=25 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:38:21.476975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=6 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:38:28.908155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323f6f0-1f5d-467f-8aae-48af356d8f6c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:38:28.923425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323f6f0-1f5d-467f-8aae-48af356d8f6c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:38:28.929407 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e34e2-16e1-4122-8db6-69f0d0102a58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:38:28.930090 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e34e2-16e1-4122-8db6-69f0d0102a58 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:38:28.930398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e34e2-16e1-4122-8db6-69f0d0102a58 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:38:28.930703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e34e2-16e1-4122-8db6-69f0d0102a58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:38:28.932503 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1d3e34e2-16e1-4122-8db6-69f0d0102a58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:38:28.946957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e34e2-16e1-4122-8db6-69f0d0102a58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:38:28.948613 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e34e2-16e1-4122-8db6-69f0d0102a58 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:38:28.961689 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1275/2590] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:38:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:38:35.276538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90c6d1-cf9e-4938-b124-bc9cc45ec3bf service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:38:35.278441 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9f90c6d1-cf9e-4938-b124-bc9cc45ec3bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:38:35.300594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90c6d1-cf9e-4938-b124-bc9cc45ec3bf service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:38:35.309744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc1b6c-6040-4331-96fe-89fc43c4a4cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:38:35.314280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc1b6c-6040-4331-96fe-89fc43c4a4cd service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:38:35.314280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc1b6c-6040-4331-96fe-89fc43c4a4cd service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:38:35.314280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc1b6c-6040-4331-96fe-89fc43c4a4cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:38:35.315718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-23fc1b6c-6040-4331-96fe-89fc43c4a4cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:38:35.339699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc1b6c-6040-4331-96fe-89fc43c4a4cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:38:35.343137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc1b6c-6040-4331-96fe-89fc43c4a4cd service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:38:35.365616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1316/2591] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:38:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:38:38.956496 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=6 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:38:45.355232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=6 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:10.640927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e34e2-16e1-4122-8db6-69f0d0102a58 service nova] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:10.650875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e34e2-16e1-4122-8db6-69f0d0102a58 service nova] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:10.655747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9f2a6-7b3c-4f2d-92fa-a8403432392d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:10.656368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f2a6-7b3c-4f2d-92fa-a8403432392d service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:10.656672 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f2a6-7b3c-4f2d-92fa-a8403432392d service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:10.656938 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f2a6-7b3c-4f2d-92fa-a8403432392d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:10.659206 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f1f9f2a6-7b3c-4f2d-92fa-a8403432392d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:10.748063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9f2a6-7b3c-4f2d-92fa-a8403432392d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:10.749684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9f2a6-7b3c-4f2d-92fa-a8403432392d service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:10.760920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1276/2592] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:15.423265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc1b6c-6040-4331-96fe-89fc43c4a4cd service nova] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:15.425111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-23fc1b6c-6040-4331-96fe-89fc43c4a4cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:15.447660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc1b6c-6040-4331-96fe-89fc43c4a4cd service nova] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:15.453811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:15.454501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:15.454781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:15.455056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] PATH_INFO: `/v3/users/864d7f47381e4e4498ad110dbfcfc154` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:15.456270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=864d7f47381e4e4498ad110dbfcfc154)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:15.458258 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:15.679035 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '864d7f47381e4e4498ad110dbfcfc154'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:15.679731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '864d7f47381e4e4498ad110dbfcfc154'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:15.680343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '864d7f47381e4e4498ad110dbfcfc154'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:15.759031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '864d7f47381e4e4498ad110dbfcfc154'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:15.767821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-2ee9871c-163a-472d-a4cc-a3964ff20632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '864d7f47381e4e4498ad110dbfcfc154'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:15.788237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1317/2593] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:39:15 2022] DELETE /identity/v3/users/864d7f47381e4e4498ad110dbfcfc154 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:16.100568 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9f2a6-7b3c-4f2d-92fa-a8403432392d service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:16.117928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9f2a6-7b3c-4f2d-92fa-a8403432392d service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:16.189933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:16.190602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:16.190886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:16.191152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] PATH_INFO: `/v3/projects/c20e7f3aca4c49ae84390a3f1a8ce410` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:16.192440 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20e7f3aca4c49ae84390a3f1a8ce410)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:16.194410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:16.312895 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20e7f3aca4c49ae84390a3f1a8ce410', 'request_id': 'req-3b9654af-b59a-46a0-97a9-c1878b7994e9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:16.319970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20e7f3aca4c49ae84390a3f1a8ce410', 'request_id': 'req-3b9654af-b59a-46a0-97a9-c1878b7994e9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:16.395199 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20e7f3aca4c49ae84390a3f1a8ce410', 'request_id': 'req-3b9654af-b59a-46a0-97a9-c1878b7994e9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:16.408996 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] The token cache is being invalidate because project c20e7f3aca4c49ae84390a3f1a8ce410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:39:16.409268 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:16.409971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-3b9654af-b59a-46a0-97a9-c1878b7994e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:16.411612 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1277/2594] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:39:16 2022] DELETE /identity/v3/projects/c20e7f3aca4c49ae84390a3f1a8ce410 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:16.416848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61d08b-2e58-4adf-addd-a4ec2daff159 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:16.417122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61d08b-2e58-4adf-addd-a4ec2daff159 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:16.417402 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61d08b-2e58-4adf-addd-a4ec2daff159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:16.439027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-5c61d08b-2e58-4adf-addd-a4ec2daff159 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:16.440293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5c61d08b-2e58-4adf-addd-a4ec2daff159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:16.510040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1318/2595] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:16.520679 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e724a9a-deb3-4bc4-b0a0-980fc9b22df6 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:16.520998 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e724a9a-deb3-4bc4-b0a0-980fc9b22df6 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:16.521220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e724a9a-deb3-4bc4-b0a0-980fc9b22df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:16.542200 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-7e724a9a-deb3-4bc4-b0a0-980fc9b22df6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:16.544606 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7e724a9a-deb3-4bc4-b0a0-980fc9b22df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:16.553298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1278/2596] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:16.558078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61d08b-2e58-4adf-addd-a4ec2daff159 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:16.569424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61d08b-2e58-4adf-addd-a4ec2daff159 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:16.575854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041868a-4150-4330-9f73-ed026ba965ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:16.576654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041868a-4150-4330-9f73-ed026ba965ed admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:16.577155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041868a-4150-4330-9f73-ed026ba965ed admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:16.577564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041868a-4150-4330-9f73-ed026ba965ed admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:16.578435 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041868a-4150-4330-9f73-ed026ba965ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:39:16.578973 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041868a-4150-4330-9f73-ed026ba965ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:16.580943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041868a-4150-4330-9f73-ed026ba965ed admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:16.595004 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1319/2597] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:39:16.599857 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e724a9a-deb3-4bc4-b0a0-980fc9b22df6 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:16.611207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e724a9a-deb3-4bc4-b0a0-980fc9b22df6 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:16.616198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789989c-bdca-4609-98d8-9450e09e9130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:16.616813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789989c-bdca-4609-98d8-9450e09e9130 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:16.617089 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789989c-bdca-4609-98d8-9450e09e9130 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:16.617462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789989c-bdca-4609-98d8-9450e09e9130 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:16.618120 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789989c-bdca-4609-98d8-9450e09e9130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:16.620158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789989c-bdca-4609-98d8-9450e09e9130 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:16.644740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:16.769899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1279/2598] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:39:16 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:39:16.772361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041868a-4150-4330-9f73-ed026ba965ed admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:16.791829 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041868a-4150-4330-9f73-ed026ba965ed admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:16.795156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789989c-bdca-4609-98d8-9450e09e9130 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:16.795156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7789989c-bdca-4609-98d8-9450e09e9130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:16.869193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c6225-11db-46df-8fff-f4753833b74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:16.869804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c6225-11db-46df-8fff-f4753833b74b admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:16.870085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c6225-11db-46df-8fff-f4753833b74b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:16.870381 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c6225-11db-46df-8fff-f4753833b74b admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:16.870531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789989c-bdca-4609-98d8-9450e09e9130 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:16.871046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c6225-11db-46df-8fff-f4753833b74b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:16.873703 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c6225-11db-46df-8fff-f4753833b74b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:16.874985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47252d32-5cac-456f-807d-79c62b8a1713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:16.875600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47252d32-5cac-456f-807d-79c62b8a1713 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:16.875880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47252d32-5cac-456f-807d-79c62b8a1713 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:16.876147 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47252d32-5cac-456f-807d-79c62b8a1713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:16.878224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-47252d32-5cac-456f-807d-79c62b8a1713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:16.882124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:16.894290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47252d32-5cac-456f-807d-79c62b8a1713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:16.895956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47252d32-5cac-456f-807d-79c62b8a1713 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:16.908218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1280/2599] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:39:16.978040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1320/2600] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:39:16 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:39:16.982671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47252d32-5cac-456f-807d-79c62b8a1713 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:16.996229 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47252d32-5cac-456f-807d-79c62b8a1713 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:17.005540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595858f7-36e3-43cc-8127-30bb2e8fea48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:17.006539 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595858f7-36e3-43cc-8127-30bb2e8fea48 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:17.007008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595858f7-36e3-43cc-8127-30bb2e8fea48 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:17.007441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595858f7-36e3-43cc-8127-30bb2e8fea48 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:17.008435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595858f7-36e3-43cc-8127-30bb2e8fea48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:17.011155 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595858f7-36e3-43cc-8127-30bb2e8fea48 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:17.030064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1281/2601] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:39:17.032367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c6225-11db-46df-8fff-f4753833b74b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:17.054802 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c6225-11db-46df-8fff-f4753833b74b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:17.057789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd35f4-e252-4b2b-a854-0c8fd3949c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:17.058412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd35f4-e252-4b2b-a854-0c8fd3949c8f admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:17.058800 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd35f4-e252-4b2b-a854-0c8fd3949c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:17.059819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd35f4-e252-4b2b-a854-0c8fd3949c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:17.060946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd35f4-e252-4b2b-a854-0c8fd3949c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:17.063969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd35f4-e252-4b2b-a854-0c8fd3949c8f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:17.087637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1321/2602] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:17.088927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595858f7-36e3-43cc-8127-30bb2e8fea48 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:17.100455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595858f7-36e3-43cc-8127-30bb2e8fea48 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:17.107137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f2277-2dc9-4c1d-a6d0-39ef9b9480f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:17.108091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2277-2dc9-4c1d-a6d0-39ef9b9480f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:17.108682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2277-2dc9-4c1d-a6d0-39ef9b9480f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:17.109301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2277-2dc9-4c1d-a6d0-39ef9b9480f7 admin admin] PATH_INFO: `/v3/projects/5f7d9ac076ce40b09043ced00f54f33a/users/5042520b122a4c6c94321a50ea57633a/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:17.139141 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f2277-2dc9-4c1d-a6d0-39ef9b9480f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7d9ac076ce40b09043ced00f54f33a, user_id=5042520b122a4c6c94321a50ea57633a, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:17.143740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f2277-2dc9-4c1d-a6d0-39ef9b9480f7 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:17.258058 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1282/2603] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:39:17 2022] PUT /identity/v3/projects/5f7d9ac076ce40b09043ced00f54f33a/users/5042520b122a4c6c94321a50ea57633a/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:17.265042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963956c5-9d34-442e-a252-9499161ac743 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:17.265852 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963956c5-9d34-442e-a252-9499161ac743 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:17.266241 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963956c5-9d34-442e-a252-9499161ac743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:17.295354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-963956c5-9d34-442e-a252-9499161ac743 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:17.296952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-963956c5-9d34-442e-a252-9499161ac743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:17.382365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1322/2604] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:17.397617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f2277-2dc9-4c1d-a6d0-39ef9b9480f7 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:17.476402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f2277-2dc9-4c1d-a6d0-39ef9b9480f7 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:17.480364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7527d12-7e10-4aa8-992f-da7f2840648f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:17.481370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527d12-7e10-4aa8-992f-da7f2840648f service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:17.481930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527d12-7e10-4aa8-992f-da7f2840648f service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:17.482418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527d12-7e10-4aa8-992f-da7f2840648f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:17.502087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7527d12-7e10-4aa8-992f-da7f2840648f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:17.503773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7527d12-7e10-4aa8-992f-da7f2840648f service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:17.515988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1283/2605] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:22.227283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963956c5-9d34-442e-a252-9499161ac743 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:22.229702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-963956c5-9d34-442e-a252-9499161ac743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:22.249039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963956c5-9d34-442e-a252-9499161ac743 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:22.263630 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:22.265625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:22.267084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:22.267618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] PATH_INFO: `/v3/users/b0db362075b04b2294a6268576a5bba3` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:22.272147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0db362075b04b2294a6268576a5bba3)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:22.276697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:22.535106 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0db362075b04b2294a6268576a5bba3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:22.537250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0db362075b04b2294a6268576a5bba3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:22.537945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0db362075b04b2294a6268576a5bba3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:22.663772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0db362075b04b2294a6268576a5bba3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:22.673610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-36696d59-7ddd-4fb7-a05d-aac2bbf7a54e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0db362075b04b2294a6268576a5bba3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:22.703842 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1323/2606] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:39:22 2022] DELETE /identity/v3/users/b0db362075b04b2294a6268576a5bba3 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:23.297072 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7527d12-7e10-4aa8-992f-da7f2840648f service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:23.332560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7527d12-7e10-4aa8-992f-da7f2840648f service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:23.478674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:23.480574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:23.481084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:23.482169 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] PATH_INFO: `/v3/projects/6436020e938047408d1f979e23dd48da` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:23.484339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6436020e938047408d1f979e23dd48da)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:23.487859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:23.504874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:23.683270 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6436020e938047408d1f979e23dd48da', 'request_id': 'req-d2b5d5de-7536-43a7-af47-76bd538610fc'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:23.693460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6436020e938047408d1f979e23dd48da', 'request_id': 'req-d2b5d5de-7536-43a7-af47-76bd538610fc'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:23.778547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6436020e938047408d1f979e23dd48da', 'request_id': 'req-d2b5d5de-7536-43a7-af47-76bd538610fc'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:23.793801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] The token cache is being invalidate because project 6436020e938047408d1f979e23dd48da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:39:23.794768 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:23.795953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d2b5d5de-7536-43a7-af47-76bd538610fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:23.798813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1284/2607] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:39:23 2022] DELETE /identity/v3/projects/6436020e938047408d1f979e23dd48da => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:23.803540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b5afa-2e22-4ab0-a54f-6fe337a4e999 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:23.804091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b5afa-2e22-4ab0-a54f-6fe337a4e999 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:23.804091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b5afa-2e22-4ab0-a54f-6fe337a4e999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:23.833577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d31b5afa-2e22-4ab0-a54f-6fe337a4e999 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:23.835069 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d31b5afa-2e22-4ab0-a54f-6fe337a4e999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:23.911196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:23.942452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1324/2608] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:23.953535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf4329-f62e-434c-934f-a16beedcf97e None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:23.953827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf4329-f62e-434c-934f-a16beedcf97e None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:23.954067 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf4329-f62e-434c-934f-a16beedcf97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:23.982636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-70bf4329-f62e-434c-934f-a16beedcf97e None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:23.984395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-70bf4329-f62e-434c-934f-a16beedcf97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:23.997143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1285/2609] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:24.002576 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b5afa-2e22-4ab0-a54f-6fe337a4e999 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:24.017946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b5afa-2e22-4ab0-a54f-6fe337a4e999 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:24.023147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d1ff0-6d49-436c-922c-f1cf0a2f5d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:24.023988 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d1ff0-6d49-436c-922c-f1cf0a2f5d45 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:24.024270 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d1ff0-6d49-436c-922c-f1cf0a2f5d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:24.024546 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d1ff0-6d49-436c-922c-f1cf0a2f5d45 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:24.025293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d1ff0-6d49-436c-922c-f1cf0a2f5d45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:39:24.025799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d1ff0-6d49-436c-922c-f1cf0a2f5d45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:24.028514 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d1ff0-6d49-436c-922c-f1cf0a2f5d45 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:24.054341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1325/2610] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:39:24.064063 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf4329-f62e-434c-934f-a16beedcf97e None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:24.079834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf4329-f62e-434c-934f-a16beedcf97e None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:24.089247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4a276-8466-403f-86ef-ea9f0c400858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:24.091831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4a276-8466-403f-86ef-ea9f0c400858 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:24.092314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4a276-8466-403f-86ef-ea9f0c400858 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:24.092796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4a276-8466-403f-86ef-ea9f0c400858 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:24.093984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4a276-8466-403f-86ef-ea9f0c400858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:24.098091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4a276-8466-403f-86ef-ea9f0c400858 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:24.237595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1286/2611] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:39:24 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:39:24.239555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d1ff0-6d49-436c-922c-f1cf0a2f5d45 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:24.257505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d1ff0-6d49-436c-922c-f1cf0a2f5d45 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:24.335668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873f85-397d-4e9d-80cc-4da0b36c56ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:24.336615 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873f85-397d-4e9d-80cc-4da0b36c56ca admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:24.337100 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873f85-397d-4e9d-80cc-4da0b36c56ca admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:24.337573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873f85-397d-4e9d-80cc-4da0b36c56ca admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:24.338760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2873f85-397d-4e9d-80cc-4da0b36c56ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:24.343134 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2873f85-397d-4e9d-80cc-4da0b36c56ca admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:24.496445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1326/2612] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:39:24 2022] POST /identity/v3/users => generated 290 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:39:24.496639 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4a276-8466-403f-86ef-ea9f0c400858 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:24.515174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4a276-8466-403f-86ef-ea9f0c400858 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:24.522932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01654c-664c-4855-8eda-bf6fe2bd985b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:24.524553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01654c-664c-4855-8eda-bf6fe2bd985b admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:24.524553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01654c-664c-4855-8eda-bf6fe2bd985b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:24.531136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01654c-664c-4855-8eda-bf6fe2bd985b admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:24.531136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01654c-664c-4855-8eda-bf6fe2bd985b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:24.531136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01654c-664c-4855-8eda-bf6fe2bd985b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:24.546535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1287/2613] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:24.548126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873f85-397d-4e9d-80cc-4da0b36c56ca admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:24.566066 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873f85-397d-4e9d-80cc-4da0b36c56ca admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:24.572091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef97a9b-e432-4f3b-932c-52325a31c06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:24.572601 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef97a9b-e432-4f3b-932c-52325a31c06d admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:24.572908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef97a9b-e432-4f3b-932c-52325a31c06d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:24.573181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef97a9b-e432-4f3b-932c-52325a31c06d admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:24.575792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef97a9b-e432-4f3b-932c-52325a31c06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:24.577789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef97a9b-e432-4f3b-932c-52325a31c06d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:24.601320 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01654c-664c-4855-8eda-bf6fe2bd985b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:24.603076 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1327/2614] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:24.614376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01654c-664c-4855-8eda-bf6fe2bd985b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:24.620483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d626ec4-67f5-4111-ba8e-712b0cf669f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:24.622074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d626ec4-67f5-4111-ba8e-712b0cf669f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:24.622386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d626ec4-67f5-4111-ba8e-712b0cf669f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:24.622726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d626ec4-67f5-4111-ba8e-712b0cf669f0 admin admin] PATH_INFO: `/v3/projects/c26653b2ecdf47fa8024b8b5470f5196/users/70a2d48e324948ffa2ccac79af7bf8b3/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:24.643418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d626ec4-67f5-4111-ba8e-712b0cf669f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26653b2ecdf47fa8024b8b5470f5196, user_id=70a2d48e324948ffa2ccac79af7bf8b3, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:24.646385 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d626ec4-67f5-4111-ba8e-712b0cf669f0 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:24.751338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1288/2615] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:39:24 2022] PUT /identity/v3/projects/c26653b2ecdf47fa8024b8b5470f5196/users/70a2d48e324948ffa2ccac79af7bf8b3/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:24.766965 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21be1d5-0c62-492c-9591-a31746cf8955 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:24.767316 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21be1d5-0c62-492c-9591-a31746cf8955 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:24.767968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21be1d5-0c62-492c-9591-a31746cf8955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:24.809422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f21be1d5-0c62-492c-9591-a31746cf8955 None None] MFA Rules not processed for user `70a2d48e324948ffa2ccac79af7bf8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:24.811506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f21be1d5-0c62-492c-9591-a31746cf8955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:24.918917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1328/2616] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:24.924950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d626ec4-67f5-4111-ba8e-712b0cf669f0 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:25.052888 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d626ec4-67f5-4111-ba8e-712b0cf669f0 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:25.061442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122b3d5-45ed-4bd4-a497-40b8ef05f68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:25.062097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122b3d5-45ed-4bd4-a497-40b8ef05f68b admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:25.062370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122b3d5-45ed-4bd4-a497-40b8ef05f68b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:25.062852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122b3d5-45ed-4bd4-a497-40b8ef05f68b admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:25.063403 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122b3d5-45ed-4bd4-a497-40b8ef05f68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:25.067485 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122b3d5-45ed-4bd4-a497-40b8ef05f68b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:25.212712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1289/2617] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:39:24 2022] POST /identity/v3/projects => generated 381 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:39:25.219624 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21be1d5-0c62-492c-9591-a31746cf8955 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:25.254020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21be1d5-0c62-492c-9591-a31746cf8955 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:25.327560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:25.346748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be399e4-f25c-40b2-bb1b-85d2fff31a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:25.351701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be399e4-f25c-40b2-bb1b-85d2fff31a8f admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:25.351701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be399e4-f25c-40b2-bb1b-85d2fff31a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:25.351701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be399e4-f25c-40b2-bb1b-85d2fff31a8f admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:25.351701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be399e4-f25c-40b2-bb1b-85d2fff31a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:25.351701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be399e4-f25c-40b2-bb1b-85d2fff31a8f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:25.546145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1329/2618] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:39:25 2022] POST /identity/v3/users => generated 297 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:39:25.551433 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122b3d5-45ed-4bd4-a497-40b8ef05f68b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:25.575323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122b3d5-45ed-4bd4-a497-40b8ef05f68b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:25.585362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4182b8e-d418-49d8-99a2-962b4b0ffe75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:25.586719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4182b8e-d418-49d8-99a2-962b4b0ffe75 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:25.587044 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4182b8e-d418-49d8-99a2-962b4b0ffe75 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:25.587788 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4182b8e-d418-49d8-99a2-962b4b0ffe75 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:25.589029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4182b8e-d418-49d8-99a2-962b4b0ffe75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:25.592970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4182b8e-d418-49d8-99a2-962b4b0ffe75 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:25.654411 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1290/2619] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:25.657549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be399e4-f25c-40b2-bb1b-85d2fff31a8f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:25.671242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be399e4-f25c-40b2-bb1b-85d2fff31a8f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:25.677016 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03932dab-6552-48ee-b242-e1f6ecf1be0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:25.677705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03932dab-6552-48ee-b242-e1f6ecf1be0e admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:25.678006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03932dab-6552-48ee-b242-e1f6ecf1be0e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:25.678306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03932dab-6552-48ee-b242-e1f6ecf1be0e admin admin] PATH_INFO: `/v3/projects/3515443a8b784a63bd459763a349688f/users/791b8d7931fe43929f0d7cec7e8fbbf6/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:25.725347 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03932dab-6552-48ee-b242-e1f6ecf1be0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3515443a8b784a63bd459763a349688f, user_id=791b8d7931fe43929f0d7cec7e8fbbf6, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:25.728547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03932dab-6552-48ee-b242-e1f6ecf1be0e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:25.860571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8330a-c61d-4f22-9300-3d02da040e06 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:25.862747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8330a-c61d-4f22-9300-3d02da040e06 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:25.862747 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8330a-c61d-4f22-9300-3d02da040e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:25.876907 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:25.888309 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1330/2620] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:39:25 2022] PUT /identity/v3/projects/3515443a8b784a63bd459763a349688f/users/791b8d7931fe43929f0d7cec7e8fbbf6/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:25.893845 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03932dab-6552-48ee-b242-e1f6ecf1be0e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:25.926919 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-48c8330a-c61d-4f22-9300-3d02da040e06 None None] MFA Rules not processed for user `5042520b122a4c6c94321a50ea57633a`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:25.929195 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-48c8330a-c61d-4f22-9300-3d02da040e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:26.013015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03932dab-6552-48ee-b242-e1f6ecf1be0e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:26.031804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578e4bf-f668-4aff-bf5c-c8b3a5c76029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:26.032859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578e4bf-f668-4aff-bf5c-c8b3a5c76029 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:26.033524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578e4bf-f668-4aff-bf5c-c8b3a5c76029 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:26.040490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578e4bf-f668-4aff-bf5c-c8b3a5c76029 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:26.040490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578e4bf-f668-4aff-bf5c-c8b3a5c76029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:26.040490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578e4bf-f668-4aff-bf5c-c8b3a5c76029 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:26.052964 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1331/2621] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:26.057027 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578e4bf-f668-4aff-bf5c-c8b3a5c76029 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:26.060428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1291/2622] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:39:26.070132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578e4bf-f668-4aff-bf5c-c8b3a5c76029 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:26.076560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1116340-d8ef-41da-9a43-1e478a238bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:26.077240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116340-d8ef-41da-9a43-1e478a238bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:26.077535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116340-d8ef-41da-9a43-1e478a238bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:26.077965 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116340-d8ef-41da-9a43-1e478a238bdf admin admin] PATH_INFO: `/v3/projects/3515443a8b784a63bd459763a349688f/users/791b8d7931fe43929f0d7cec7e8fbbf6/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:26.080167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1116340-d8ef-41da-9a43-1e478a238bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3515443a8b784a63bd459763a349688f, user_id=791b8d7931fe43929f0d7cec7e8fbbf6, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:26.081140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8330a-c61d-4f22-9300-3d02da040e06 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:26.082633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-48c8330a-c61d-4f22-9300-3d02da040e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:26.083687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1116340-d8ef-41da-9a43-1e478a238bdf admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:26.097873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1332/2623] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:39:26 2022] PUT /identity/v3/projects/3515443a8b784a63bd459763a349688f/users/791b8d7931fe43929f0d7cec7e8fbbf6/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:39:26.111263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae1f54-2a40-4116-b15c-fb17cf321476 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:26.111548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae1f54-2a40-4116-b15c-fb17cf321476 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:26.111815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae1f54-2a40-4116-b15c-fb17cf321476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:26.145553 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-c2ae1f54-2a40-4116-b15c-fb17cf321476 None None] MFA Rules not processed for user `791b8d7931fe43929f0d7cec7e8fbbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:26.146928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c2ae1f54-2a40-4116-b15c-fb17cf321476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:26.213426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8330a-c61d-4f22-9300-3d02da040e06 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:26.248447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1333/2624] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:26.281716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae1f54-2a40-4116-b15c-fb17cf321476 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:26.284639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-c2ae1f54-2a40-4116-b15c-fb17cf321476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:26.321454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:26.347769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b1a68-c9eb-4803-8f79-b0c0c4560eb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:26.348442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b1a68-c9eb-4803-8f79-b0c0c4560eb7 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:26.348766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b1a68-c9eb-4803-8f79-b0c0c4560eb7 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:26.348966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b1a68-c9eb-4803-8f79-b0c0c4560eb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:26.383791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae1f54-2a40-4116-b15c-fb17cf321476 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:26.383791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318da43c-950f-4945-8452-38fff84c735f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:26.384210 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318da43c-950f-4945-8452-38fff84c735f service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:26.384600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318da43c-950f-4945-8452-38fff84c735f service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:26.385054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318da43c-950f-4945-8452-38fff84c735f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:26.396357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:26.399366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318da43c-950f-4945-8452-38fff84c735f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:26.401024 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318da43c-950f-4945-8452-38fff84c735f service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:26.415582 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1334/2625] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:26.438843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b1a68-c9eb-4803-8f79-b0c0c4560eb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:26.440500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b1a68-c9eb-4803-8f79-b0c0c4560eb7 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:26.456765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1292/2626] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:39:26.701415 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318da43c-950f-4945-8452-38fff84c735f service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:26.717200 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318da43c-950f-4945-8452-38fff84c735f service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:26.724127 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a4236-6dbf-4183-9d4b-26f838d1d22e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:26.724941 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a4236-6dbf-4183-9d4b-26f838d1d22e service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:26.725428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a4236-6dbf-4183-9d4b-26f838d1d22e service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:26.725894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a4236-6dbf-4183-9d4b-26f838d1d22e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:26.839040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a4236-6dbf-4183-9d4b-26f838d1d22e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:26.842278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a4236-6dbf-4183-9d4b-26f838d1d22e service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:26.859175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1335/2627] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:30.126810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b89912-76c7-4b6f-ac70-c7cc72144a63 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:30.127540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b89912-76c7-4b6f-ac70-c7cc72144a63 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:30.127706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b89912-76c7-4b6f-ac70-c7cc72144a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:30.164967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a8b89912-76c7-4b6f-ac70-c7cc72144a63 None None] MFA Rules not processed for user `5042520b122a4c6c94321a50ea57633a`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:30.167508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a8b89912-76c7-4b6f-ac70-c7cc72144a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:30.182381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1293/2628] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:39:30.603235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a4236-6dbf-4183-9d4b-26f838d1d22e service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:30.604901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-af3a4236-6dbf-4183-9d4b-26f838d1d22e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:30.681846 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a4236-6dbf-4183-9d4b-26f838d1d22e service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:30.687454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d136b-c782-4904-88b0-24a20544ec0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:30.688525 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d136b-c782-4904-88b0-24a20544ec0d service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:30.689031 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d136b-c782-4904-88b0-24a20544ec0d service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:30.689494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d136b-c782-4904-88b0-24a20544ec0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:30.692511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-094d136b-c782-4904-88b0-24a20544ec0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:30.708688 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d136b-c782-4904-88b0-24a20544ec0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:30.710361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d136b-c782-4904-88b0-24a20544ec0d service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:30.720754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1336/2629] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:36.552448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b89912-76c7-4b6f-ac70-c7cc72144a63 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:36.554603 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a8b89912-76c7-4b6f-ac70-c7cc72144a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:36.636914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b89912-76c7-4b6f-ac70-c7cc72144a63 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:36.642058 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27428f82-37f0-4b82-ac7c-92cf6b2f4687 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:36.642697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428f82-37f0-4b82-ac7c-92cf6b2f4687 service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:36.642978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428f82-37f0-4b82-ac7c-92cf6b2f4687 service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:36.643253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428f82-37f0-4b82-ac7c-92cf6b2f4687 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:36.645008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-27428f82-37f0-4b82-ac7c-92cf6b2f4687 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:36.659545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27428f82-37f0-4b82-ac7c-92cf6b2f4687 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:36.661471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27428f82-37f0-4b82-ac7c-92cf6b2f4687 service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:36.673234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1294/2630] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:36.675998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d136b-c782-4904-88b0-24a20544ec0d service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:36.677467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-094d136b-c782-4904-88b0-24a20544ec0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:36.694724 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d136b-c782-4904-88b0-24a20544ec0d service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:36.701157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fca3b2-cd97-481f-8d20-eb58763513ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:36.701782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fca3b2-cd97-481f-8d20-eb58763513ba service placement] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:36.702060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fca3b2-cd97-481f-8d20-eb58763513ba service placement] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:36.702326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fca3b2-cd97-481f-8d20-eb58763513ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:36.713457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fca3b2-cd97-481f-8d20-eb58763513ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:36.715900 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fca3b2-cd97-481f-8d20-eb58763513ba service placement] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:36.727293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1337/2631] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:42.107256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27428f82-37f0-4b82-ac7c-92cf6b2f4687 service placement] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:42.180570 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27428f82-37f0-4b82-ac7c-92cf6b2f4687 service placement] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:42.187070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:42.187846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:42.188220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:42.188642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] PATH_INFO: `/v3/users/70a2d48e324948ffa2ccac79af7bf8b3` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:42.190029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] RBAC: Authorizing `identity:delete_user(user_id=70a2d48e324948ffa2ccac79af7bf8b3)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:42.192191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:42.398105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70a2d48e324948ffa2ccac79af7bf8b3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:42.398799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70a2d48e324948ffa2ccac79af7bf8b3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:42.399415 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70a2d48e324948ffa2ccac79af7bf8b3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:42.408339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70a2d48e324948ffa2ccac79af7bf8b3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:42.409543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:42.422628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70a2d48e324948ffa2ccac79af7bf8b3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:42.520811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1295/2632] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:39:42 2022] DELETE /identity/v3/users/70a2d48e324948ffa2ccac79af7bf8b3 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:42.525429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc88bc-3559-45cd-8c39-b1e07031b988 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:42.525987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc88bc-3559-45cd-8c39-b1e07031b988 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:42.525987 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc88bc-3559-45cd-8c39-b1e07031b988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:42.547051 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-b8cc88bc-3559-45cd-8c39-b1e07031b988 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:42.548294 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b8cc88bc-3559-45cd-8c39-b1e07031b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:42.600056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:42.620148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1338/2633] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:42.633060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:42.634543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:42.702521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848dd8b8-35d6-42d1-9eab-bb8c2af7abba admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:42.707631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2e15f-3051-475d-98e9-026e9bf385df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:42.708339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e15f-3051-475d-98e9-026e9bf385df service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:42.708690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e15f-3051-475d-98e9-026e9bf385df service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:42.709048 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e15f-3051-475d-98e9-026e9bf385df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:42.722358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2e15f-3051-475d-98e9-026e9bf385df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:42.724191 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2e15f-3051-475d-98e9-026e9bf385df service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:42.737569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1296/2634] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:43.242355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc88bc-3559-45cd-8c39-b1e07031b988 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:43.244183 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-b8cc88bc-3559-45cd-8c39-b1e07031b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:43.263878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc88bc-3559-45cd-8c39-b1e07031b988 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:43.272299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:43.273233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:43.273723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:43.274178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] PATH_INFO: `/v3/projects/c26653b2ecdf47fa8024b8b5470f5196` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:43.277614 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26653b2ecdf47fa8024b8b5470f5196)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:43.281307 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:43.421290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26653b2ecdf47fa8024b8b5470f5196', 'request_id': 'req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:43.428232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26653b2ecdf47fa8024b8b5470f5196', 'request_id': 'req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:43.435356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26653b2ecdf47fa8024b8b5470f5196', 'request_id': 'req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:43.531757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] The token cache is being invalidate because project c26653b2ecdf47fa8024b8b5470f5196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:39:43.532413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:43.532635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:43.534184 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1339/2635] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:39:43 2022] DELETE /identity/v3/projects/c26653b2ecdf47fa8024b8b5470f5196 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:43.538256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2e15f-3051-475d-98e9-026e9bf385df service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:43.548087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2e15f-3051-475d-98e9-026e9bf385df service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:43.612325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:43.612984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:43.613267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:43.613540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] PATH_INFO: `/v3/users/791b8d7931fe43929f0d7cec7e8fbbf6` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:43.614821 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=791b8d7931fe43929f0d7cec7e8fbbf6)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:43.616868 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:43.824752 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791b8d7931fe43929f0d7cec7e8fbbf6'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:43.825817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791b8d7931fe43929f0d7cec7e8fbbf6'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:43.826825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791b8d7931fe43929f0d7cec7e8fbbf6'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:43.839642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791b8d7931fe43929f0d7cec7e8fbbf6'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:43.855844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-640a4bbd-8457-4a84-8a34-f86f5d427fc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791b8d7931fe43929f0d7cec7e8fbbf6'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:43.860175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:43.947188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1297/2636] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:39:43 2022] DELETE /identity/v3/users/791b8d7931fe43929f0d7cec7e8fbbf6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:44.510610 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:44.512564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:44.527306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfab85c-90ae-402d-93b5-dc7aeee99c9f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:44.596929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:44.598405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:44.598764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:44.599037 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] PATH_INFO: `/v3/projects/3515443a8b784a63bd459763a349688f` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:44.600297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3515443a8b784a63bd459763a349688f)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:44.602466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:44.772257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3515443a8b784a63bd459763a349688f', 'request_id': 'req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:44.779986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3515443a8b784a63bd459763a349688f', 'request_id': 'req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:44.787145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3515443a8b784a63bd459763a349688f', 'request_id': 'req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:44.885312 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] The token cache is being invalidate because project 3515443a8b784a63bd459763a349688f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:39:44.885609 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:44.887135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-a7aeb571-dc3a-45dd-81ec-6638444f0fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:44.888692 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1340/2637] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:39:44 2022] DELETE /identity/v3/projects/3515443a8b784a63bd459763a349688f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:44.893886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeab38-2be2-42a1-bcc8-074f53c14f51 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:44.894710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeab38-2be2-42a1-bcc8-074f53c14f51 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:44.894710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeab38-2be2-42a1-bcc8-074f53c14f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:44.918361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-03eeab38-2be2-42a1-bcc8-074f53c14f51 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:44.919673 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-03eeab38-2be2-42a1-bcc8-074f53c14f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:44.995326 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1298/2638] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:45.004337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbd655-8c73-4180-831d-93851593cfaf None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:45.004623 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbd655-8c73-4180-831d-93851593cfaf None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:45.004893 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbd655-8c73-4180-831d-93851593cfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:45.018940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:45.040486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-cccbd655-8c73-4180-831d-93851593cfaf None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:45.041699 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-cccbd655-8c73-4180-831d-93851593cfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:45.060862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1341/2639] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:45.060971 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeab38-2be2-42a1-bcc8-074f53c14f51 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:45.070775 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeab38-2be2-42a1-bcc8-074f53c14f51 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:45.078587 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061902f-bfa3-44e9-a34c-7a204147ee27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:45.079239 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061902f-bfa3-44e9-a34c-7a204147ee27 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:45.079520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061902f-bfa3-44e9-a34c-7a204147ee27 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:45.079800 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061902f-bfa3-44e9-a34c-7a204147ee27 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:45.080471 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061902f-bfa3-44e9-a34c-7a204147ee27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:39:45.080770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061902f-bfa3-44e9-a34c-7a204147ee27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:45.082521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061902f-bfa3-44e9-a34c-7a204147ee27 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:45.098130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1299/2640] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:39:45.104266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbd655-8c73-4180-831d-93851593cfaf None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:45.118021 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbd655-8c73-4180-831d-93851593cfaf None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:45.125368 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6630ae-8f2d-4200-b400-862368ba7255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:45.126559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6630ae-8f2d-4200-b400-862368ba7255 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:45.126840 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6630ae-8f2d-4200-b400-862368ba7255 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:45.127111 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6630ae-8f2d-4200-b400-862368ba7255 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:45.127804 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6630ae-8f2d-4200-b400-862368ba7255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:45.129843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6630ae-8f2d-4200-b400-862368ba7255 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:45.242531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1342/2641] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:39:45 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:39:45.247841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061902f-bfa3-44e9-a34c-7a204147ee27 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:45.264813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061902f-bfa3-44e9-a34c-7a204147ee27 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:45.326742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5c16f-0eb0-4712-b71a-661baa3ccfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:45.327264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5c16f-0eb0-4712-b71a-661baa3ccfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:45.327540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5c16f-0eb0-4712-b71a-661baa3ccfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:45.327849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5c16f-0eb0-4712-b71a-661baa3ccfc5 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:45.328477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5c16f-0eb0-4712-b71a-661baa3ccfc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:45.330438 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5c16f-0eb0-4712-b71a-661baa3ccfc5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:45.443523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1300/2642] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:39:45 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:39:45.447324 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6630ae-8f2d-4200-b400-862368ba7255 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:45.460376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6630ae-8f2d-4200-b400-862368ba7255 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:45.466056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233f92e-41cc-42a3-a0c5-db03e21c9568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:45.466708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233f92e-41cc-42a3-a0c5-db03e21c9568 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:45.466984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233f92e-41cc-42a3-a0c5-db03e21c9568 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:45.467254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233f92e-41cc-42a3-a0c5-db03e21c9568 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:45.468209 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233f92e-41cc-42a3-a0c5-db03e21c9568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:45.469826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233f92e-41cc-42a3-a0c5-db03e21c9568 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:45.481192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1343/2643] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:45.485545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5c16f-0eb0-4712-b71a-661baa3ccfc5 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:45.499221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5c16f-0eb0-4712-b71a-661baa3ccfc5 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:45.504254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d116913-6d43-4f57-a400-fba061bf30c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:45.504863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d116913-6d43-4f57-a400-fba061bf30c1 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:45.505135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d116913-6d43-4f57-a400-fba061bf30c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:45.505405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d116913-6d43-4f57-a400-fba061bf30c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:45.506043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d116913-6d43-4f57-a400-fba061bf30c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:45.507695 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d116913-6d43-4f57-a400-fba061bf30c1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:45.520644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1301/2644] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:45.525562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233f92e-41cc-42a3-a0c5-db03e21c9568 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:45.544625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233f92e-41cc-42a3-a0c5-db03e21c9568 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:45.549253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9ff71-5775-4c9e-a46a-ce91a577cc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:45.549914 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9ff71-5775-4c9e-a46a-ce91a577cc92 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:45.550191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9ff71-5775-4c9e-a46a-ce91a577cc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:45.550495 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9ff71-5775-4c9e-a46a-ce91a577cc92 admin admin] PATH_INFO: `/v3/projects/42b332dedf024928be26fab346f17200/users/ab2006be3c524c1fbafa9fde37362d2f/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:45.569923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9ff71-5775-4c9e-a46a-ce91a577cc92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b332dedf024928be26fab346f17200, user_id=ab2006be3c524c1fbafa9fde37362d2f, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:45.572715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9ff71-5775-4c9e-a46a-ce91a577cc92 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:45.662924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1344/2645] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:39:45 2022] PUT /identity/v3/projects/42b332dedf024928be26fab346f17200/users/ab2006be3c524c1fbafa9fde37362d2f/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:45.674932 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad46cb-9ba1-4a8a-aafa-a18176b62e48 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:45.675219 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad46cb-9ba1-4a8a-aafa-a18176b62e48 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:45.675476 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad46cb-9ba1-4a8a-aafa-a18176b62e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:45.706628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-08ad46cb-9ba1-4a8a-aafa-a18176b62e48 None None] MFA Rules not processed for user `ab2006be3c524c1fbafa9fde37362d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:45.707905 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-08ad46cb-9ba1-4a8a-aafa-a18176b62e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:45.759148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:45.779285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1302/2646] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:45.784003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9ff71-5775-4c9e-a46a-ce91a577cc92 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:45.875397 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9ff71-5775-4c9e-a46a-ce91a577cc92 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:45.883019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae750fd-4857-4e7e-9acb-af1a227897dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:45.883773 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae750fd-4857-4e7e-9acb-af1a227897dd admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:45.884054 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae750fd-4857-4e7e-9acb-af1a227897dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:45.884327 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae750fd-4857-4e7e-9acb-af1a227897dd admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:45.884983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae750fd-4857-4e7e-9acb-af1a227897dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:45.887997 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae750fd-4857-4e7e-9acb-af1a227897dd admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:45.978153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1345/2647] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:39:45 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:39:45.983285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad46cb-9ba1-4a8a-aafa-a18176b62e48 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:45.996643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad46cb-9ba1-4a8a-aafa-a18176b62e48 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:46.065222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b71336-79ae-4c28-bc91-fc6b527a5572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:46.065839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b71336-79ae-4c28-bc91-fc6b527a5572 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:46.066158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b71336-79ae-4c28-bc91-fc6b527a5572 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:46.066443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b71336-79ae-4c28-bc91-fc6b527a5572 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:46.067129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b71336-79ae-4c28-bc91-fc6b527a5572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:46.069122 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b71336-79ae-4c28-bc91-fc6b527a5572 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:46.199775 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1303/2648] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:39:45 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:39:46.203912 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae750fd-4857-4e7e-9acb-af1a227897dd admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:46.215343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae750fd-4857-4e7e-9acb-af1a227897dd admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:46.219995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca877e-4b71-4f14-8457-30d4480c3b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:46.220801 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca877e-4b71-4f14-8457-30d4480c3b44 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:46.221012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca877e-4b71-4f14-8457-30d4480c3b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:46.221283 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca877e-4b71-4f14-8457-30d4480c3b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:46.221928 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca877e-4b71-4f14-8457-30d4480c3b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:46.223585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca877e-4b71-4f14-8457-30d4480c3b44 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:46.235555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1346/2649] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:46.240054 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b71336-79ae-4c28-bc91-fc6b527a5572 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:46.251796 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b71336-79ae-4c28-bc91-fc6b527a5572 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:46.256781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cf9e8-3aea-46bc-b023-88ed1fb0d5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:46.257460 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cf9e8-3aea-46bc-b023-88ed1fb0d5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:46.257737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cf9e8-3aea-46bc-b023-88ed1fb0d5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:46.258009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cf9e8-3aea-46bc-b023-88ed1fb0d5b9 admin admin] PATH_INFO: `/v3/projects/c52125db23c14adfb72aaa804411d404/users/70e56abc3d12454b8d0e48a2f953760e/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:46.275602 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655cf9e8-3aea-46bc-b023-88ed1fb0d5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52125db23c14adfb72aaa804411d404, user_id=70e56abc3d12454b8d0e48a2f953760e, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:46.278659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655cf9e8-3aea-46bc-b023-88ed1fb0d5b9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:46.361413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1304/2650] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:39:46 2022] PUT /identity/v3/projects/c52125db23c14adfb72aaa804411d404/users/70e56abc3d12454b8d0e48a2f953760e/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:46.365967 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca877e-4b71-4f14-8457-30d4480c3b44 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:46.383478 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca877e-4b71-4f14-8457-30d4480c3b44 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:46.460012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577492d-ed3c-4baf-b9eb-f61c13a368b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:46.460833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577492d-ed3c-4baf-b9eb-f61c13a368b3 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:46.461225 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577492d-ed3c-4baf-b9eb-f61c13a368b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:46.461676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577492d-ed3c-4baf-b9eb-f61c13a368b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:46.462961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577492d-ed3c-4baf-b9eb-f61c13a368b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:46.466173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577492d-ed3c-4baf-b9eb-f61c13a368b3 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:46.483233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1347/2651] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:46.487831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cf9e8-3aea-46bc-b023-88ed1fb0d5b9 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:46.502739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cf9e8-3aea-46bc-b023-88ed1fb0d5b9 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:46.507942 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7d5a8-e854-4385-ba95-9cda3ef6b1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:46.508619 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7d5a8-e854-4385-ba95-9cda3ef6b1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:46.508898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7d5a8-e854-4385-ba95-9cda3ef6b1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:46.509189 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7d5a8-e854-4385-ba95-9cda3ef6b1d4 admin admin] PATH_INFO: `/v3/projects/c52125db23c14adfb72aaa804411d404/users/70e56abc3d12454b8d0e48a2f953760e/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:46.511249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7d5a8-e854-4385-ba95-9cda3ef6b1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52125db23c14adfb72aaa804411d404, user_id=70e56abc3d12454b8d0e48a2f953760e, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:46.514066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7d5a8-e854-4385-ba95-9cda3ef6b1d4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:46.525864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1305/2652] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:39:46 2022] PUT /identity/v3/projects/c52125db23c14adfb72aaa804411d404/users/70e56abc3d12454b8d0e48a2f953760e/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:46.536945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806dd5f-b5e3-47de-a7ee-46455c6808cf None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:46.537212 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806dd5f-b5e3-47de-a7ee-46455c6808cf None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:46.537486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806dd5f-b5e3-47de-a7ee-46455c6808cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:46.562237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:46.571219 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-8806dd5f-b5e3-47de-a7ee-46455c6808cf None None] MFA Rules not processed for user `70e56abc3d12454b8d0e48a2f953760e`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:46.572499 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8806dd5f-b5e3-47de-a7ee-46455c6808cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:46.788782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1348/2653] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:46.795425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7d5a8-e854-4385-ba95-9cda3ef6b1d4 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:46.797306 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0be7d5a8-e854-4385-ba95-9cda3ef6b1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:46.867923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7d5a8-e854-4385-ba95-9cda3ef6b1d4 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:46.872310 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f30cef-2fdf-48bd-b256-7299f9da9d41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:46.873079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f30cef-2fdf-48bd-b256-7299f9da9d41 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:46.873462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f30cef-2fdf-48bd-b256-7299f9da9d41 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:46.873844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f30cef-2fdf-48bd-b256-7299f9da9d41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:46.885419 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f30cef-2fdf-48bd-b256-7299f9da9d41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:46.887671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f30cef-2fdf-48bd-b256-7299f9da9d41 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:46.898817 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1306/2654] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:47.217608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806dd5f-b5e3-47de-a7ee-46455c6808cf None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:47.234816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806dd5f-b5e3-47de-a7ee-46455c6808cf None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:47.241033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7489e49-f532-4b2b-b0cd-36bd6f41a98d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:47.241972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7489e49-f532-4b2b-b0cd-36bd6f41a98d service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:47.242460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7489e49-f532-4b2b-b0cd-36bd6f41a98d service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:47.242979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7489e49-f532-4b2b-b0cd-36bd6f41a98d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:47.325161 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7489e49-f532-4b2b-b0cd-36bd6f41a98d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:47.327502 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7489e49-f532-4b2b-b0cd-36bd6f41a98d service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:47.339962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1349/2655] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:51.192059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f30cef-2fdf-48bd-b256-7299f9da9d41 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:51.260032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:51.277395 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f30cef-2fdf-48bd-b256-7299f9da9d41 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:51.282305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:51.285598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:51.285598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:51.285598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] PATH_INFO: `/v3/users/ab2006be3c524c1fbafa9fde37362d2f` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:51.285598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2006be3c524c1fbafa9fde37362d2f)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:51.287004 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:51.499069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2006be3c524c1fbafa9fde37362d2f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:51.499769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2006be3c524c1fbafa9fde37362d2f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:51.500469 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2006be3c524c1fbafa9fde37362d2f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:51.509965 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2006be3c524c1fbafa9fde37362d2f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:51.532064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2006be3c524c1fbafa9fde37362d2f'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:51.629261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1307/2656] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:39:51 2022] DELETE /identity/v3/users/ab2006be3c524c1fbafa9fde37362d2f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:51.634831 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e7127-ab19-45a1-8cb7-8a9215b9903a None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:51.635326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e7127-ab19-45a1-8cb7-8a9215b9903a None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:51.635574 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e7127-ab19-45a1-8cb7-8a9215b9903a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:51.661257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2e7e7127-ab19-45a1-8cb7-8a9215b9903a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:51.662944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2e7e7127-ab19-45a1-8cb7-8a9215b9903a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:51.752607 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1350/2657] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:51.759955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:51.763807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:51.888977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c82cc0-c699-4c8b-88ee-aba398a241d1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:51.894302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca27cd-2fe3-465e-8272-304110112f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:51.895533 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca27cd-2fe3-465e-8272-304110112f7e service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:51.896116 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca27cd-2fe3-465e-8272-304110112f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:51.896686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca27cd-2fe3-465e-8272-304110112f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:51.916538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca27cd-2fe3-465e-8272-304110112f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:51.918990 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca27cd-2fe3-465e-8272-304110112f7e service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:51.939690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1308/2658] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:39:52.824486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e7127-ab19-45a1-8cb7-8a9215b9903a None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:52.828330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2e7e7127-ab19-45a1-8cb7-8a9215b9903a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:52.852459 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e7127-ab19-45a1-8cb7-8a9215b9903a None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:52.865715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:52.865715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:52.865715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:52.865715 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] PATH_INFO: `/v3/projects/42b332dedf024928be26fab346f17200` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:52.867698 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b332dedf024928be26fab346f17200)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:52.872807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:53.061230 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:53.082269 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b332dedf024928be26fab346f17200', 'request_id': 'req-e319f163-ca8b-4eea-87bb-106d74557ef3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:53.092733 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b332dedf024928be26fab346f17200', 'request_id': 'req-e319f163-ca8b-4eea-87bb-106d74557ef3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:53.100971 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b332dedf024928be26fab346f17200', 'request_id': 'req-e319f163-ca8b-4eea-87bb-106d74557ef3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:53.192596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] The token cache is being invalidate because project 42b332dedf024928be26fab346f17200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:39:53.193095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:53.194214 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:53.196930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1351/2659] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:39:52 2022] DELETE /identity/v3/projects/42b332dedf024928be26fab346f17200 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:53.203802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca27cd-2fe3-465e-8272-304110112f7e service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:53.218456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca27cd-2fe3-465e-8272-304110112f7e service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:53.321757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:53.322797 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:53.324174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:53.324609 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] PATH_INFO: `/v3/users/70e56abc3d12454b8d0e48a2f953760e` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:53.326700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70e56abc3d12454b8d0e48a2f953760e)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:53.330316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:53.346927 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:53.538804 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70e56abc3d12454b8d0e48a2f953760e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:53.539531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70e56abc3d12454b8d0e48a2f953760e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:53.540277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70e56abc3d12454b8d0e48a2f953760e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:53.549340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70e56abc3d12454b8d0e48a2f953760e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:53.566134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-ca62b44f-24ae-4976-9e2a-8bdd57ad0014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70e56abc3d12454b8d0e48a2f953760e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:53.661657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1309/2660] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:39:53 2022] DELETE /identity/v3/users/70e56abc3d12454b8d0e48a2f953760e => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:54.145732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:54.153042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:54.169708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319f163-ca8b-4eea-87bb-106d74557ef3 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:54.280958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:54.281605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:54.281887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:54.282165 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] PATH_INFO: `/v3/projects/c52125db23c14adfb72aaa804411d404` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:54.283739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c52125db23c14adfb72aaa804411d404)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:54.285729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:54.420733 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c52125db23c14adfb72aaa804411d404', 'request_id': 'req-38f1ee42-5011-4109-8b98-2c30728aa242'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:54.428226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c52125db23c14adfb72aaa804411d404', 'request_id': 'req-38f1ee42-5011-4109-8b98-2c30728aa242'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:54.436067 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c52125db23c14adfb72aaa804411d404', 'request_id': 'req-38f1ee42-5011-4109-8b98-2c30728aa242'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:54.542884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] The token cache is being invalidate because project c52125db23c14adfb72aaa804411d404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:39:54.543439 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:54.544163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-38f1ee42-5011-4109-8b98-2c30728aa242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:39:54.545755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1352/2661] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:39:54 2022] DELETE /identity/v3/projects/c52125db23c14adfb72aaa804411d404 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:54.551463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74c8f-92ab-4dd0-b3a8-faf472a157b0 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:54.551463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74c8f-92ab-4dd0-b3a8-faf472a157b0 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:54.552427 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74c8f-92ab-4dd0-b3a8-faf472a157b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:54.574435 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d7c74c8f-92ab-4dd0-b3a8-faf472a157b0 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:54.575726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d7c74c8f-92ab-4dd0-b3a8-faf472a157b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:54.650302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1310/2662] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:54.659375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7612c2-1c63-4bfd-8bc5-a68a91436e69 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:54.659709 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7612c2-1c63-4bfd-8bc5-a68a91436e69 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:54.659908 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7612c2-1c63-4bfd-8bc5-a68a91436e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:54.681370 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-bc7612c2-1c63-4bfd-8bc5-a68a91436e69 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:54.683335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bc7612c2-1c63-4bfd-8bc5-a68a91436e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:54.692701 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1353/2663] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:39:54.696985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74c8f-92ab-4dd0-b3a8-faf472a157b0 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:54.711064 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74c8f-92ab-4dd0-b3a8-faf472a157b0 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:54.716331 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbf8fc-76cf-4ca9-ae7c-e263bf0ef911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:54.716995 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbf8fc-76cf-4ca9-ae7c-e263bf0ef911 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:54.717318 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbf8fc-76cf-4ca9-ae7c-e263bf0ef911 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:54.717700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbf8fc-76cf-4ca9-ae7c-e263bf0ef911 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:54.718461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbf8fc-76cf-4ca9-ae7c-e263bf0ef911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:39:54.718894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbf8fc-76cf-4ca9-ae7c-e263bf0ef911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:54.720677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbf8fc-76cf-4ca9-ae7c-e263bf0ef911 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:54.734887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1311/2664] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:39:54.739841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7612c2-1c63-4bfd-8bc5-a68a91436e69 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:54.751767 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7612c2-1c63-4bfd-8bc5-a68a91436e69 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:54.756848 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256bc82-6bc6-49f6-9171-26752674f255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:54.757473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256bc82-6bc6-49f6-9171-26752674f255 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:54.757942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256bc82-6bc6-49f6-9171-26752674f255 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:54.758218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256bc82-6bc6-49f6-9171-26752674f255 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:54.758911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256bc82-6bc6-49f6-9171-26752674f255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:54.761272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256bc82-6bc6-49f6-9171-26752674f255 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:54.887581 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1354/2665] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:39:54 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:39:54.892128 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbf8fc-76cf-4ca9-ae7c-e263bf0ef911 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:54.903609 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbf8fc-76cf-4ca9-ae7c-e263bf0ef911 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:54.966638 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ae012-4372-4145-9c70-4be20f177370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:54.967185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ae012-4372-4145-9c70-4be20f177370 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:54.967465 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ae012-4372-4145-9c70-4be20f177370 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:54.967741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ae012-4372-4145-9c70-4be20f177370 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:54.968383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ae012-4372-4145-9c70-4be20f177370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:54.970373 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ae012-4372-4145-9c70-4be20f177370 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:55.064323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1312/2666] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:39:54 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:39:55.068492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256bc82-6bc6-49f6-9171-26752674f255 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:55.080392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256bc82-6bc6-49f6-9171-26752674f255 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:55.085137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3739e2e-3e98-4071-8455-4a065de6468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:55.085794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739e2e-3e98-4071-8455-4a065de6468b admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:55.090871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739e2e-3e98-4071-8455-4a065de6468b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:55.090871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739e2e-3e98-4071-8455-4a065de6468b admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:55.090871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3739e2e-3e98-4071-8455-4a065de6468b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:55.090871 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3739e2e-3e98-4071-8455-4a065de6468b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:55.100796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1355/2667] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:55.104980 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ae012-4372-4145-9c70-4be20f177370 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:55.111711 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:55.115618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ae012-4372-4145-9c70-4be20f177370 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:55.121259 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236c62f-fac9-4996-8be2-9c3b42f14baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:55.127156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236c62f-fac9-4996-8be2-9c3b42f14baf admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:55.127156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236c62f-fac9-4996-8be2-9c3b42f14baf admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:55.127156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236c62f-fac9-4996-8be2-9c3b42f14baf admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:55.127156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236c62f-fac9-4996-8be2-9c3b42f14baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:55.127156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236c62f-fac9-4996-8be2-9c3b42f14baf admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:55.137324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1313/2668] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:55.141604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3739e2e-3e98-4071-8455-4a065de6468b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:55.152757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3739e2e-3e98-4071-8455-4a065de6468b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:55.157771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ed4ba-a60c-4994-8b01-479bb1cfa5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:55.158465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ed4ba-a60c-4994-8b01-479bb1cfa5ab admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:55.161549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ed4ba-a60c-4994-8b01-479bb1cfa5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:55.161549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ed4ba-a60c-4994-8b01-479bb1cfa5ab admin admin] PATH_INFO: `/v3/projects/b825f1551eb24bc39cac55e7534623b3/users/83fb82cd6cd64fb1ad2c87b077bb5f08/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:55.162569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:55.186841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ed4ba-a60c-4994-8b01-479bb1cfa5ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b825f1551eb24bc39cac55e7534623b3, user_id=83fb82cd6cd64fb1ad2c87b077bb5f08, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:55.189750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ed4ba-a60c-4994-8b01-479bb1cfa5ab admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:55.273659 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1356/2669] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:39:55 2022] PUT /identity/v3/projects/b825f1551eb24bc39cac55e7534623b3/users/83fb82cd6cd64fb1ad2c87b077bb5f08/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:55.286302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6362a-4011-4e8e-9c53-810626e61203 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:55.286636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6362a-4011-4e8e-9c53-810626e61203 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:55.286896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6362a-4011-4e8e-9c53-810626e61203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:55.318757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-44e6362a-4011-4e8e-9c53-810626e61203 None None] MFA Rules not processed for user `83fb82cd6cd64fb1ad2c87b077bb5f08`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:55.320002 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-44e6362a-4011-4e8e-9c53-810626e61203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:55.394190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1314/2670] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:39:55.399490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ed4ba-a60c-4994-8b01-479bb1cfa5ab admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:55.472105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ed4ba-a60c-4994-8b01-479bb1cfa5ab admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:55.477463 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb1395-9c00-41e2-9f2a-009c47c8bb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:55.478095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb1395-9c00-41e2-9f2a-009c47c8bb9c admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:55.478369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb1395-9c00-41e2-9f2a-009c47c8bb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:55.478667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb1395-9c00-41e2-9f2a-009c47c8bb9c admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:55.480631 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb1395-9c00-41e2-9f2a-009c47c8bb9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:55.482664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb1395-9c00-41e2-9f2a-009c47c8bb9c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:55.584297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1357/2671] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:39:55 2022] POST /identity/v3/projects => generated 367 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:39:55.589015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6362a-4011-4e8e-9c53-810626e61203 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:55.600849 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6362a-4011-4e8e-9c53-810626e61203 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:55.674508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baea5c3-1f8e-4b97-bd31-e961573a9209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:55.675260 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baea5c3-1f8e-4b97-bd31-e961573a9209 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:55.675543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baea5c3-1f8e-4b97-bd31-e961573a9209 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:55.675829 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baea5c3-1f8e-4b97-bd31-e961573a9209 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:55.676463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baea5c3-1f8e-4b97-bd31-e961573a9209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:55.678513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baea5c3-1f8e-4b97-bd31-e961573a9209 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:55.851873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1315/2672] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:39:55 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:39:55.856588 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb1395-9c00-41e2-9f2a-009c47c8bb9c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:55.867254 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb1395-9c00-41e2-9f2a-009c47c8bb9c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:55.872190 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267dd74-3341-4a96-8dd2-d6f17cdf2bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:55.872808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267dd74-3341-4a96-8dd2-d6f17cdf2bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:55.873084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267dd74-3341-4a96-8dd2-d6f17cdf2bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:55.873352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267dd74-3341-4a96-8dd2-d6f17cdf2bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:55.873999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267dd74-3341-4a96-8dd2-d6f17cdf2bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:55.875671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267dd74-3341-4a96-8dd2-d6f17cdf2bb8 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:55.888091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1358/2673] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:55.899860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baea5c3-1f8e-4b97-bd31-e961573a9209 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:55.920545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baea5c3-1f8e-4b97-bd31-e961573a9209 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:55.928222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b343a-fe38-48f6-873d-0c242ed75e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:55.929320 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b343a-fe38-48f6-873d-0c242ed75e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:55.929815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b343a-fe38-48f6-873d-0c242ed75e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:55.930315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b343a-fe38-48f6-873d-0c242ed75e5f admin admin] PATH_INFO: `/v3/projects/6e5f0cacdc0144c48386a6f31923b2b1/users/ef9f89988cd040e88d79e0d3bec9450b/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:55.960172 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b343a-fe38-48f6-873d-0c242ed75e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5f0cacdc0144c48386a6f31923b2b1, user_id=ef9f89988cd040e88d79e0d3bec9450b, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:55.966897 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b343a-fe38-48f6-873d-0c242ed75e5f admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:56.108858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1316/2674] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:39:55 2022] PUT /identity/v3/projects/6e5f0cacdc0144c48386a6f31923b2b1/users/ef9f89988cd040e88d79e0d3bec9450b/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:56.115418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267dd74-3341-4a96-8dd2-d6f17cdf2bb8 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:56.127072 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267dd74-3341-4a96-8dd2-d6f17cdf2bb8 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:56.191462 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650fa73-f985-491b-a06a-952b7a800e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:56.192206 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650fa73-f985-491b-a06a-952b7a800e60 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:56.192343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650fa73-f985-491b-a06a-952b7a800e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:56.192599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650fa73-f985-491b-a06a-952b7a800e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:56.193333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650fa73-f985-491b-a06a-952b7a800e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:56.196278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650fa73-f985-491b-a06a-952b7a800e60 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:56.208543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1359/2675] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:39:56.218261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b343a-fe38-48f6-873d-0c242ed75e5f admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:56.230640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b343a-fe38-48f6-873d-0c242ed75e5f admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:56.235860 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d29673-8bdd-4314-9312-0c8f7edd62d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:56.236561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d29673-8bdd-4314-9312-0c8f7edd62d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:56.236833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d29673-8bdd-4314-9312-0c8f7edd62d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:56.237107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d29673-8bdd-4314-9312-0c8f7edd62d5 admin admin] PATH_INFO: `/v3/projects/6e5f0cacdc0144c48386a6f31923b2b1/users/ef9f89988cd040e88d79e0d3bec9450b/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:56.239188 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d29673-8bdd-4314-9312-0c8f7edd62d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5f0cacdc0144c48386a6f31923b2b1, user_id=ef9f89988cd040e88d79e0d3bec9450b, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:56.242035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d29673-8bdd-4314-9312-0c8f7edd62d5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:56.255413 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1317/2676] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:39:56 2022] PUT /identity/v3/projects/6e5f0cacdc0144c48386a6f31923b2b1/users/ef9f89988cd040e88d79e0d3bec9450b/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:39:56.267124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df60469-1a4b-447b-90d2-120f2bdd8300 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:56.267526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df60469-1a4b-447b-90d2-120f2bdd8300 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:56.267840 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df60469-1a4b-447b-90d2-120f2bdd8300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:56.299673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-2df60469-1a4b-447b-90d2-120f2bdd8300 None None] MFA Rules not processed for user `ef9f89988cd040e88d79e0d3bec9450b`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:39:56.300898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-2df60469-1a4b-447b-90d2-120f2bdd8300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:56.350423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:56.378878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1360/2677] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:39:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:39:56.391281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d29673-8bdd-4314-9312-0c8f7edd62d5 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:39:56.393132 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-20d29673-8bdd-4314-9312-0c8f7edd62d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:39:56.440798 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:39:56.461748 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d29673-8bdd-4314-9312-0c8f7edd62d5 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:39:56.465757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f5df1-b609-4662-a604-75cb37c29842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:39:56.466386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f5df1-b609-4662-a604-75cb37c29842 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:39:56.466691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f5df1-b609-4662-a604-75cb37c29842 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:39:56.466961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f5df1-b609-4662-a604-75cb37c29842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:39:56.476993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f5df1-b609-4662-a604-75cb37c29842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:39:56.478697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f5df1-b609-4662-a604-75cb37c29842 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:39:56.489450 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1318/2678] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:39:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:39:59.968090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df60469-1a4b-447b-90d2-120f2bdd8300 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:00.083536 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df60469-1a4b-447b-90d2-120f2bdd8300 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:00.089166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:00.089962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:00.090342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:00.090846 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] PATH_INFO: `/v3/users/83fb82cd6cd64fb1ad2c87b077bb5f08` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:00.092350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83fb82cd6cd64fb1ad2c87b077bb5f08)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:00.095745 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:00.303867 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83fb82cd6cd64fb1ad2c87b077bb5f08'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:00.304577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83fb82cd6cd64fb1ad2c87b077bb5f08'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:00.305217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83fb82cd6cd64fb1ad2c87b077bb5f08'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:00.376239 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83fb82cd6cd64fb1ad2c87b077bb5f08'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:00.391976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83fb82cd6cd64fb1ad2c87b077bb5f08'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:00.421398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1361/2679] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:39:59 2022] DELETE /identity/v3/users/83fb82cd6cd64fb1ad2c87b077bb5f08 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:00.426952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0ca93-fda4-46ba-b29e-3224f16b059d None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:00.427802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0ca93-fda4-46ba-b29e-3224f16b059d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:00.428003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0ca93-fda4-46ba-b29e-3224f16b059d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:00.450728 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b3e0ca93-fda4-46ba-b29e-3224f16b059d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:00.452029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b3e0ca93-fda4-46ba-b29e-3224f16b059d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:00.525286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1319/2680] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:00.539907 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:00.541846 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:00.615387 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b59be5-50a3-444a-a22a-383a82461a39 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:00.619531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac6892-4474-47f2-a040-4aedce317f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:00.620195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac6892-4474-47f2-a040-4aedce317f67 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:00.620482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac6892-4474-47f2-a040-4aedce317f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:00.620920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac6892-4474-47f2-a040-4aedce317f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:00.632095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac6892-4474-47f2-a040-4aedce317f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:00.635015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac6892-4474-47f2-a040-4aedce317f67 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:00.655568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1362/2681] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:40:01.162399 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0ca93-fda4-46ba-b29e-3224f16b059d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:01.164853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b3e0ca93-fda4-46ba-b29e-3224f16b059d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:01.184398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0ca93-fda4-46ba-b29e-3224f16b059d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:01.192296 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:01.193408 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:01.193874 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:01.194326 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] PATH_INFO: `/v3/projects/b825f1551eb24bc39cac55e7534623b3` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:01.196621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b825f1551eb24bc39cac55e7534623b3)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:01.202092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:01.340387 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b825f1551eb24bc39cac55e7534623b3', 'request_id': 'req-bccce96c-915c-46d1-9354-69a4204fd3fc'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:01.348060 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b825f1551eb24bc39cac55e7534623b3', 'request_id': 'req-bccce96c-915c-46d1-9354-69a4204fd3fc'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:01.422036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b825f1551eb24bc39cac55e7534623b3', 'request_id': 'req-bccce96c-915c-46d1-9354-69a4204fd3fc'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:01.434801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] The token cache is being invalidate because project b825f1551eb24bc39cac55e7534623b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:40:01.435518 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:01.440203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:01.440203 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1320/2682] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:40:01 2022] DELETE /identity/v3/projects/b825f1551eb24bc39cac55e7534623b3 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:01.442235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac6892-4474-47f2-a040-4aedce317f67 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:01.448440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:01.454196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac6892-4474-47f2-a040-4aedce317f67 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:01.531399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:01.532108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:01.532472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:01.532787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] PATH_INFO: `/v3/users/ef9f89988cd040e88d79e0d3bec9450b` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:01.534105 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef9f89988cd040e88d79e0d3bec9450b)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:01.536652 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:01.748482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef9f89988cd040e88d79e0d3bec9450b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:01.749994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef9f89988cd040e88d79e0d3bec9450b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:01.751559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef9f89988cd040e88d79e0d3bec9450b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:01.873811 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef9f89988cd040e88d79e0d3bec9450b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:01.883718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0e2fb5dd-9c24-4c40-b8e9-36a46eda0725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef9f89988cd040e88d79e0d3bec9450b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:01.910147 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1363/2683] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:40:01 2022] DELETE /identity/v3/users/ef9f89988cd040e88d79e0d3bec9450b => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:40:02.382594 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:02.384560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:02.404127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccce96c-915c-46d1-9354-69a4204fd3fc admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:02.476943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:02.478370 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:02.478801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:02.480890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] PATH_INFO: `/v3/projects/6e5f0cacdc0144c48386a6f31923b2b1` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:02.483357 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5f0cacdc0144c48386a6f31923b2b1)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:02.487070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:02.490221 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:02.664626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5f0cacdc0144c48386a6f31923b2b1', 'request_id': 'req-94d638c5-8c40-411a-bc93-7c3b2bc719f1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:02.671757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5f0cacdc0144c48386a6f31923b2b1', 'request_id': 'req-94d638c5-8c40-411a-bc93-7c3b2bc719f1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:02.787737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5f0cacdc0144c48386a6f31923b2b1', 'request_id': 'req-94d638c5-8c40-411a-bc93-7c3b2bc719f1'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:02.801653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] The token cache is being invalidate because project 6e5f0cacdc0144c48386a6f31923b2b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:40:02.802100 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:02.802918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-94d638c5-8c40-411a-bc93-7c3b2bc719f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:02.804574 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1321/2684] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:40:02 2022] DELETE /identity/v3/projects/6e5f0cacdc0144c48386a6f31923b2b1 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:02.810694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2992dd-ac32-4093-9851-cb3ca867cc9c None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:02.811242 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2992dd-ac32-4093-9851-cb3ca867cc9c None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:02.811457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2992dd-ac32-4093-9851-cb3ca867cc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:02.834674 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6a2992dd-ac32-4093-9851-cb3ca867cc9c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:02.835944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6a2992dd-ac32-4093-9851-cb3ca867cc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:02.914418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1364/2685] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:02.923690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebaeb9-04c9-4676-a2a0-3fcb397d0c3a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:02.924074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebaeb9-04c9-4676-a2a0-3fcb397d0c3a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:02.924223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebaeb9-04c9-4676-a2a0-3fcb397d0c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:02.946087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-35ebaeb9-04c9-4676-a2a0-3fcb397d0c3a None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:02.948852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-35ebaeb9-04c9-4676-a2a0-3fcb397d0c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:02.966962 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1322/2686] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:02.968859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2992dd-ac32-4093-9851-cb3ca867cc9c None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:02.987093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2992dd-ac32-4093-9851-cb3ca867cc9c None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:02.992058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1e5ca-e3fb-4996-b495-13a15ebba7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:02.992677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1e5ca-e3fb-4996-b495-13a15ebba7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:02.992958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1e5ca-e3fb-4996-b495-13a15ebba7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:02.993221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1e5ca-e3fb-4996-b495-13a15ebba7d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:02.993877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1e5ca-e3fb-4996-b495-13a15ebba7d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:40:02.994175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1e5ca-e3fb-4996-b495-13a15ebba7d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:02.996141 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1e5ca-e3fb-4996-b495-13a15ebba7d6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:03.009636 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1365/2687] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:40:03.014627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebaeb9-04c9-4676-a2a0-3fcb397d0c3a None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:03.031540 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebaeb9-04c9-4676-a2a0-3fcb397d0c3a None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:03.037487 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029687d0-a378-4619-bdef-e2ad456f15c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:03.038134 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029687d0-a378-4619-bdef-e2ad456f15c2 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:03.038585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029687d0-a378-4619-bdef-e2ad456f15c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:03.038882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029687d0-a378-4619-bdef-e2ad456f15c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:03.039584 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029687d0-a378-4619-bdef-e2ad456f15c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:03.042542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029687d0-a378-4619-bdef-e2ad456f15c2 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:03.139963 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1323/2688] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:40:03 2022] POST /identity/v3/projects => generated 393 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:40:03.145524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1e5ca-e3fb-4996-b495-13a15ebba7d6 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:03.158940 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1e5ca-e3fb-4996-b495-13a15ebba7d6 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:03.169193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:03.231673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042b361-88d9-4c68-b045-46be322d9472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:03.232727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042b361-88d9-4c68-b045-46be322d9472 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:03.233204 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042b361-88d9-4c68-b045-46be322d9472 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:03.233685 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042b361-88d9-4c68-b045-46be322d9472 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:03.234978 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042b361-88d9-4c68-b045-46be322d9472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:03.238985 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042b361-88d9-4c68-b045-46be322d9472 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:03.325237 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1366/2689] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:40:03 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:40:03.330344 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029687d0-a378-4619-bdef-e2ad456f15c2 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:03.342023 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029687d0-a378-4619-bdef-e2ad456f15c2 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:03.346997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f750b-5084-4931-a118-5de95b335fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:03.347630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f750b-5084-4931-a118-5de95b335fba admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:03.347913 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f750b-5084-4931-a118-5de95b335fba admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:03.348185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f750b-5084-4931-a118-5de95b335fba admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:03.348854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f750b-5084-4931-a118-5de95b335fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:03.350552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f750b-5084-4931-a118-5de95b335fba admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:03.362726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1324/2690] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:03.367271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042b361-88d9-4c68-b045-46be322d9472 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:03.378299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042b361-88d9-4c68-b045-46be322d9472 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:03.383279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6877f-c012-4b0c-981b-372a0cc941f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:03.383917 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6877f-c012-4b0c-981b-372a0cc941f6 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:03.384197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6877f-c012-4b0c-981b-372a0cc941f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:03.384470 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6877f-c012-4b0c-981b-372a0cc941f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:03.385616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6877f-c012-4b0c-981b-372a0cc941f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:03.387299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6877f-c012-4b0c-981b-372a0cc941f6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:03.398930 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1367/2691] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:40:03.404323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f750b-5084-4931-a118-5de95b335fba admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:03.415166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f750b-5084-4931-a118-5de95b335fba admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:03.420904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81690e3-ae41-4dfb-99ae-8020959c0f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:03.421547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81690e3-ae41-4dfb-99ae-8020959c0f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:03.421864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81690e3-ae41-4dfb-99ae-8020959c0f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:03.422180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81690e3-ae41-4dfb-99ae-8020959c0f45 admin admin] PATH_INFO: `/v3/projects/e377a9b6f42e4243ad1c1fd59a156250/users/a1c06a807fde45a494ef880ecfcf0d8d/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:03.447324 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81690e3-ae41-4dfb-99ae-8020959c0f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e377a9b6f42e4243ad1c1fd59a156250, user_id=a1c06a807fde45a494ef880ecfcf0d8d, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:03.452976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81690e3-ae41-4dfb-99ae-8020959c0f45 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:03.554372 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1325/2692] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:40:03 2022] PUT /identity/v3/projects/e377a9b6f42e4243ad1c1fd59a156250/users/a1c06a807fde45a494ef880ecfcf0d8d/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:03.573729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0198e9-be0f-413a-927a-8f52000fdf3f None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:03.574197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0198e9-be0f-413a-927a-8f52000fdf3f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:03.574498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0198e9-be0f-413a-927a-8f52000fdf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:03.607201 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-ef0198e9-be0f-413a-927a-8f52000fdf3f None None] MFA Rules not processed for user `a1c06a807fde45a494ef880ecfcf0d8d`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:03.608522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ef0198e9-be0f-413a-927a-8f52000fdf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:03.716765 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1368/2693] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:03.721740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81690e3-ae41-4dfb-99ae-8020959c0f45 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:03.797094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81690e3-ae41-4dfb-99ae-8020959c0f45 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:03.802104 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960bcd5-5be1-4eba-86bc-3699bca89190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:03.802760 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960bcd5-5be1-4eba-86bc-3699bca89190 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:03.803035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960bcd5-5be1-4eba-86bc-3699bca89190 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:03.803364 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960bcd5-5be1-4eba-86bc-3699bca89190 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:03.804018 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960bcd5-5be1-4eba-86bc-3699bca89190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:03.806010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960bcd5-5be1-4eba-86bc-3699bca89190 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:03.943869 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1326/2694] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:40:03 2022] POST /identity/v3/projects => generated 391 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:40:03.949115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0198e9-be0f-413a-927a-8f52000fdf3f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:03.961538 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0198e9-be0f-413a-927a-8f52000fdf3f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:04.027729 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008d27d-8c4a-4d70-94b5-0ce6714a2c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:04.028354 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008d27d-8c4a-4d70-94b5-0ce6714a2c22 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:04.028634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008d27d-8c4a-4d70-94b5-0ce6714a2c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:04.028909 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008d27d-8c4a-4d70-94b5-0ce6714a2c22 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:04.029555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008d27d-8c4a-4d70-94b5-0ce6714a2c22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:04.031559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008d27d-8c4a-4d70-94b5-0ce6714a2c22 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:04.177253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1369/2695] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:40:03 2022] POST /identity/v3/users => generated 302 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:40:04.181596 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960bcd5-5be1-4eba-86bc-3699bca89190 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:04.194368 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960bcd5-5be1-4eba-86bc-3699bca89190 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:04.200651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db6756-c844-4135-bd76-ddbb4a42d106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:04.201515 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db6756-c844-4135-bd76-ddbb4a42d106 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:04.201831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db6756-c844-4135-bd76-ddbb4a42d106 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:04.202289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db6756-c844-4135-bd76-ddbb4a42d106 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:04.203143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db6756-c844-4135-bd76-ddbb4a42d106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:04.205443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db6756-c844-4135-bd76-ddbb4a42d106 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:04.222960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1327/2696] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:04.226705 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008d27d-8c4a-4d70-94b5-0ce6714a2c22 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:04.240424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008d27d-8c4a-4d70-94b5-0ce6714a2c22 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:04.246378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1b3a8-bf33-4727-9a2e-963ae73b5624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:04.247095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1b3a8-bf33-4727-9a2e-963ae73b5624 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:04.247378 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1b3a8-bf33-4727-9a2e-963ae73b5624 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:04.247650 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1b3a8-bf33-4727-9a2e-963ae73b5624 admin admin] PATH_INFO: `/v3/projects/0596d39be965401793a44da1cf65946b/users/96b1507cd46645c0b7b65de78b356401/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:04.250513 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:04.267548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1b3a8-bf33-4727-9a2e-963ae73b5624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0596d39be965401793a44da1cf65946b, user_id=96b1507cd46645c0b7b65de78b356401, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:04.270171 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1b3a8-bf33-4727-9a2e-963ae73b5624 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:04.371329 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1370/2697] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:40:04 2022] PUT /identity/v3/projects/0596d39be965401793a44da1cf65946b/users/96b1507cd46645c0b7b65de78b356401/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:40:04.376386 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db6756-c844-4135-bd76-ddbb4a42d106 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:04.382617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:04.388975 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db6756-c844-4135-bd76-ddbb4a42d106 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:04.468770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49b25d-86c4-42f7-af74-1aee423b99ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:04.469381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49b25d-86c4-42f7-af74-1aee423b99ca admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:04.469657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49b25d-86c4-42f7-af74-1aee423b99ca admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:04.469933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49b25d-86c4-42f7-af74-1aee423b99ca admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:04.470614 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49b25d-86c4-42f7-af74-1aee423b99ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:04.472220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49b25d-86c4-42f7-af74-1aee423b99ca admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:04.482530 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1328/2698] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:40:04.486813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1b3a8-bf33-4727-9a2e-963ae73b5624 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:04.498676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1b3a8-bf33-4727-9a2e-963ae73b5624 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:04.503815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4355f1-29ae-4fbf-9fdf-50e995d30da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:04.504511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4355f1-29ae-4fbf-9fdf-50e995d30da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:04.504786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4355f1-29ae-4fbf-9fdf-50e995d30da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:04.505059 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4355f1-29ae-4fbf-9fdf-50e995d30da0 admin admin] PATH_INFO: `/v3/projects/0596d39be965401793a44da1cf65946b/users/96b1507cd46645c0b7b65de78b356401/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:04.507093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4355f1-29ae-4fbf-9fdf-50e995d30da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0596d39be965401793a44da1cf65946b, user_id=96b1507cd46645c0b7b65de78b356401, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:04.509787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4355f1-29ae-4fbf-9fdf-50e995d30da0 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:04.531641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1371/2699] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:40:04 2022] PUT /identity/v3/projects/0596d39be965401793a44da1cf65946b/users/96b1507cd46645c0b7b65de78b356401/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:04.542627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46721f8f-91f3-48f9-ab1c-e63315b344d1 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:04.542931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46721f8f-91f3-48f9-ab1c-e63315b344d1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:04.543171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46721f8f-91f3-48f9-ab1c-e63315b344d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:04.575347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-46721f8f-91f3-48f9-ab1c-e63315b344d1 None None] MFA Rules not processed for user `96b1507cd46645c0b7b65de78b356401`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:04.576615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-46721f8f-91f3-48f9-ab1c-e63315b344d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:04.657780 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1329/2700] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:04 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:04.670261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4355f1-29ae-4fbf-9fdf-50e995d30da0 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:04.672363 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-4e4355f1-29ae-4fbf-9fdf-50e995d30da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:04.741472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4355f1-29ae-4fbf-9fdf-50e995d30da0 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:04.746516 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50808c-ffb4-4509-a1fa-fb771de9f351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:04.747137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50808c-ffb4-4509-a1fa-fb771de9f351 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:04.747640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50808c-ffb4-4509-a1fa-fb771de9f351 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:04.747916 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50808c-ffb4-4509-a1fa-fb771de9f351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:04.759034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50808c-ffb4-4509-a1fa-fb771de9f351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:04.761967 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50808c-ffb4-4509-a1fa-fb771de9f351 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:04.773507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1372/2701] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:04 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:40:04.834508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46721f8f-91f3-48f9-ab1c-e63315b344d1 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:04.836069 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-46721f8f-91f3-48f9-ab1c-e63315b344d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:04.848918 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46721f8f-91f3-48f9-ab1c-e63315b344d1 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:04.855286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f64a69-e135-46d0-ad8d-710fe67a2beb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:04.855925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64a69-e135-46d0-ad8d-710fe67a2beb service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:04.856205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64a69-e135-46d0-ad8d-710fe67a2beb service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:04.856477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64a69-e135-46d0-ad8d-710fe67a2beb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:04.923008 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f64a69-e135-46d0-ad8d-710fe67a2beb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:04.924649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f64a69-e135-46d0-ad8d-710fe67a2beb service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:04.935725 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1330/2702] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:04 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:40:10.877498 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50808c-ffb4-4509-a1fa-fb771de9f351 service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:10.974637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50808c-ffb4-4509-a1fa-fb771de9f351 service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:10.980194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:10.980863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:10.981144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:10.981421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] PATH_INFO: `/v3/users/a1c06a807fde45a494ef880ecfcf0d8d` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:10.982757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1c06a807fde45a494ef880ecfcf0d8d)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:10.984728 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:11.207496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1c06a807fde45a494ef880ecfcf0d8d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:11.208486 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1c06a807fde45a494ef880ecfcf0d8d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:11.209144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1c06a807fde45a494ef880ecfcf0d8d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:11.341670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1c06a807fde45a494ef880ecfcf0d8d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:11.350249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1c06a807fde45a494ef880ecfcf0d8d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:11.361754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:11.374836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1373/2703] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:40:10 2022] DELETE /identity/v3/users/a1c06a807fde45a494ef880ecfcf0d8d => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:40:11.380103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc68854-8f69-4db5-bf8c-54f8b63a2ad0 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:11.380658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc68854-8f69-4db5-bf8c-54f8b63a2ad0 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:11.380658 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc68854-8f69-4db5-bf8c-54f8b63a2ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:11.410903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-9fc68854-8f69-4db5-bf8c-54f8b63a2ad0 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:11.413126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9fc68854-8f69-4db5-bf8c-54f8b63a2ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:11.445208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:11.508148 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1331/2704] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:11.518168 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:11.524991 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:11.593271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecd235-bc6a-4981-95fa-80cf167c9c7c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:11.597064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da7fa4-483d-4351-9145-c591eae907b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:11.597697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da7fa4-483d-4351-9145-c591eae907b3 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:11.597967 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da7fa4-483d-4351-9145-c591eae907b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:11.598238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da7fa4-483d-4351-9145-c591eae907b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:11.610226 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da7fa4-483d-4351-9145-c591eae907b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:11.611409 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da7fa4-483d-4351-9145-c591eae907b3 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:11.626384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1374/2705] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:40:12.129186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc68854-8f69-4db5-bf8c-54f8b63a2ad0 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:12.129186 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9fc68854-8f69-4db5-bf8c-54f8b63a2ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:12.167207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc68854-8f69-4db5-bf8c-54f8b63a2ad0 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:12.178401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:12.179108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:12.179383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:12.179655 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] PATH_INFO: `/v3/projects/e377a9b6f42e4243ad1c1fd59a156250` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:12.182293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e377a9b6f42e4243ad1c1fd59a156250)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:12.186412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:12.250065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4990-f1c4-43b5-976a-79284afa55de None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:12.250973 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4990-f1c4-43b5-976a-79284afa55de None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:12.251487 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4990-f1c4-43b5-976a-79284afa55de None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:12.254570 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1375/2706] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:40:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 17:40:12.261450 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99317e-4383-4f8d-9d5c-319db3b5fa0d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:12.261894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99317e-4383-4f8d-9d5c-319db3b5fa0d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:12.262295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99317e-4383-4f8d-9d5c-319db3b5fa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:12.333567 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0b99317e-4383-4f8d-9d5c-319db3b5fa0d None None] MFA Rules not processed for user `c29a91e099ba4cd5886fd910d978fb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:12.383491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0b99317e-4383-4f8d-9d5c-319db3b5fa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:12.383491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1376/2707] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:40:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:12.383491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99317e-4383-4f8d-9d5c-319db3b5fa0d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:12.383491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0b99317e-4383-4f8d-9d5c-319db3b5fa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:12.383491 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99317e-4383-4f8d-9d5c-319db3b5fa0d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:12.384386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf8622-fe96-4c52-8503-60dda269ea47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:12.384386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf8622-fe96-4c52-8503-60dda269ea47 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:12.384386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf8622-fe96-4c52-8503-60dda269ea47 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:12.384386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf8622-fe96-4c52-8503-60dda269ea47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:12.391581 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf8622-fe96-4c52-8503-60dda269ea47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:12.393489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf8622-fe96-4c52-8503-60dda269ea47 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:12.404520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1377/2708] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:40:12.526065 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acddf8-4d6f-4bd0-8660-edd8ac707f4b None None] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:12.526348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acddf8-4d6f-4bd0-8660-edd8ac707f4b None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:12.526647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acddf8-4d6f-4bd0-8660-edd8ac707f4b None None] PATH_INFO: `/` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:12.528080 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1378/2709] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 22 17:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 17:40:12.535034 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4e3df-4bf6-4e93-b7cc-1f8c15bab275 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:12.535555 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4e3df-4bf6-4e93-b7cc-1f8c15bab275 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:12.536070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4e3df-4bf6-4e93-b7cc-1f8c15bab275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:12.583056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6dd4e3df-4bf6-4e93-b7cc-1f8c15bab275 None None] MFA Rules not processed for user `9628118c1df64fa4a7ea8ce37549d8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:12.584247 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6dd4e3df-4bf6-4e93-b7cc-1f8c15bab275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:12.657874 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1379/2710] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 22 17:40:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:12.669227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4e3df-4bf6-4e93-b7cc-1f8c15bab275 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:12.679082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4e3df-4bf6-4e93-b7cc-1f8c15bab275 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:12.683543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0985a26-d2f4-4af3-884b-e796fe05adbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:12.684265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0985a26-d2f4-4af3-884b-e796fe05adbf service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:12.684541 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0985a26-d2f4-4af3-884b-e796fe05adbf service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:12.684818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0985a26-d2f4-4af3-884b-e796fe05adbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:12.697496 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0985a26-d2f4-4af3-884b-e796fe05adbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:12.699020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0985a26-d2f4-4af3-884b-e796fe05adbf service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:12.717232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1380/2711] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:40:12.965210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e377a9b6f42e4243ad1c1fd59a156250', 'request_id': 'req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:12.972542 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e377a9b6f42e4243ad1c1fd59a156250', 'request_id': 'req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:13.119768 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e377a9b6f42e4243ad1c1fd59a156250', 'request_id': 'req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:13.137870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] The token cache is being invalidate because project e377a9b6f42e4243ad1c1fd59a156250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:40:13.138376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:13.139732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:13.142452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1332/2712] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:40:12 2022] DELETE /identity/v3/projects/e377a9b6f42e4243ad1c1fd59a156250 => generated 0 bytes in 1019 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:13.148216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0985a26-d2f4-4af3-884b-e796fe05adbf service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:13.162115 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0985a26-d2f4-4af3-884b-e796fe05adbf service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:13.213571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:13.279265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:13.280384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:13.281129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:13.281752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] PATH_INFO: `/v3/users/96b1507cd46645c0b7b65de78b356401` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:13.284758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] RBAC: Authorizing `identity:delete_user(user_id=96b1507cd46645c0b7b65de78b356401)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:13.289807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:13.525666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96b1507cd46645c0b7b65de78b356401'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:13.526675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96b1507cd46645c0b7b65de78b356401'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:13.527396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96b1507cd46645c0b7b65de78b356401'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:13.605926 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96b1507cd46645c0b7b65de78b356401'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:13.614647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-c14c15a9-2109-4c8c-a80a-968aaef5b96a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96b1507cd46645c0b7b65de78b356401'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:13.646047 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1381/2713] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:40:13 2022] DELETE /identity/v3/users/96b1507cd46645c0b7b65de78b356401 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:14.223458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:14.225075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:14.239974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebe69c-21ef-4cb7-9b9f-46a2f9256c90 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:14.327065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:14.327624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:14.327902 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:14.328295 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] PATH_INFO: `/v3/projects/0596d39be965401793a44da1cf65946b` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:14.329818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0596d39be965401793a44da1cf65946b)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:14.332026 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:14.483303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:14.484741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0596d39be965401793a44da1cf65946b', 'request_id': 'req-912b99b1-0801-4dba-9e37-1099d1958739'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:14.492215 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0596d39be965401793a44da1cf65946b', 'request_id': 'req-912b99b1-0801-4dba-9e37-1099d1958739'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:14.595108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0596d39be965401793a44da1cf65946b', 'request_id': 'req-912b99b1-0801-4dba-9e37-1099d1958739'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:14.613258 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] The token cache is being invalidate because project 0596d39be965401793a44da1cf65946b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:40:14.613766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:14.615244 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-912b99b1-0801-4dba-9e37-1099d1958739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:14.622535 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1333/2714] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:40:14 2022] DELETE /identity/v3/projects/0596d39be965401793a44da1cf65946b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:14.627235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98175d-f554-496a-8e6d-6d4f41939af2 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:14.627826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98175d-f554-496a-8e6d-6d4f41939af2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:14.628245 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98175d-f554-496a-8e6d-6d4f41939af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:14.662568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-ff98175d-f554-496a-8e6d-6d4f41939af2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:14.664251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ff98175d-f554-496a-8e6d-6d4f41939af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:14.777048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1382/2715] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:14.790083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93142026-7fa3-435f-8da7-d394a4a5691f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:14.791198 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93142026-7fa3-435f-8da7-d394a4a5691f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:14.791735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93142026-7fa3-435f-8da7-d394a4a5691f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:14.815475 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-93142026-7fa3-435f-8da7-d394a4a5691f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:14.816718 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-93142026-7fa3-435f-8da7-d394a4a5691f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:14.825852 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1334/2716] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:14.830389 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98175d-f554-496a-8e6d-6d4f41939af2 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:14.841708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98175d-f554-496a-8e6d-6d4f41939af2 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:14.846621 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bdd1b-3243-419d-b15a-02cb966dc67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:14.847256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bdd1b-3243-419d-b15a-02cb966dc67b admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:14.847639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bdd1b-3243-419d-b15a-02cb966dc67b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:14.847886 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bdd1b-3243-419d-b15a-02cb966dc67b admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:14.848578 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bdd1b-3243-419d-b15a-02cb966dc67b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:40:14.848882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bdd1b-3243-419d-b15a-02cb966dc67b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:14.850469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bdd1b-3243-419d-b15a-02cb966dc67b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:14.865186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1383/2717] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:40:14.871220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93142026-7fa3-435f-8da7-d394a4a5691f None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:14.883946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93142026-7fa3-435f-8da7-d394a4a5691f None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:14.889500 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467d1b5-34d3-40ec-8374-4c23a46f1d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:14.890126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467d1b5-34d3-40ec-8374-4c23a46f1d49 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:14.890412 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467d1b5-34d3-40ec-8374-4c23a46f1d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:14.890732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467d1b5-34d3-40ec-8374-4c23a46f1d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:14.891392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467d1b5-34d3-40ec-8374-4c23a46f1d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:14.893367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467d1b5-34d3-40ec-8374-4c23a46f1d49 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:15.003129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1335/2718] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:40:14 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:40:15.007550 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bdd1b-3243-419d-b15a-02cb966dc67b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:15.018616 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bdd1b-3243-419d-b15a-02cb966dc67b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:15.057475 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:15.092255 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82b02a-b30d-4963-be1a-87724951c37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:15.092939 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82b02a-b30d-4963-be1a-87724951c37f admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:15.093272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82b02a-b30d-4963-be1a-87724951c37f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:15.093596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82b02a-b30d-4963-be1a-87724951c37f admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:15.095185 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82b02a-b30d-4963-be1a-87724951c37f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:15.097205 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82b02a-b30d-4963-be1a-87724951c37f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:15.272154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1384/2719] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:40:15 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:40:15.273052 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467d1b5-34d3-40ec-8374-4c23a46f1d49 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:15.286989 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467d1b5-34d3-40ec-8374-4c23a46f1d49 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:15.292982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdec22-7074-40f4-a9eb-ecd579b720d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:15.293621 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdec22-7074-40f4-a9eb-ecd579b720d1 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:15.293899 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdec22-7074-40f4-a9eb-ecd579b720d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:15.294171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdec22-7074-40f4-a9eb-ecd579b720d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:15.294967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdec22-7074-40f4-a9eb-ecd579b720d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:15.296759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdec22-7074-40f4-a9eb-ecd579b720d1 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:15.317812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1336/2720] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:15.321887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82b02a-b30d-4963-be1a-87724951c37f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:15.340995 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82b02a-b30d-4963-be1a-87724951c37f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:15.346983 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007c69a-974d-4e23-bda9-4fa488324517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:15.347596 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007c69a-974d-4e23-bda9-4fa488324517 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:15.347878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007c69a-974d-4e23-bda9-4fa488324517 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:15.348153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007c69a-974d-4e23-bda9-4fa488324517 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:15.348824 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007c69a-974d-4e23-bda9-4fa488324517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:15.350505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007c69a-974d-4e23-bda9-4fa488324517 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:15.369602 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1385/2721] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:40:15.374311 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdec22-7074-40f4-a9eb-ecd579b720d1 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:15.387212 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdec22-7074-40f4-a9eb-ecd579b720d1 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:15.392782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b505a-70e3-482c-814c-5236abe05565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:15.393586 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b505a-70e3-482c-814c-5236abe05565 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:15.397486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b505a-70e3-482c-814c-5236abe05565 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:15.397486 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b505a-70e3-482c-814c-5236abe05565 admin admin] PATH_INFO: `/v3/projects/7e30fc6bdbbb43bdb7fe777a35342c03/users/b4d1899602114316a2f0d6364e5e2876/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:15.416686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b505a-70e3-482c-814c-5236abe05565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e30fc6bdbbb43bdb7fe777a35342c03, user_id=b4d1899602114316a2f0d6364e5e2876, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:15.419978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b505a-70e3-482c-814c-5236abe05565 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:15.531986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1337/2722] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:40:15 2022] PUT /identity/v3/projects/7e30fc6bdbbb43bdb7fe777a35342c03/users/b4d1899602114316a2f0d6364e5e2876/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:15.544071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7252b-82e1-455d-8edf-c40705723223 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:15.544352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7252b-82e1-455d-8edf-c40705723223 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:15.544608 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7252b-82e1-455d-8edf-c40705723223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:15.585721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-05a7252b-82e1-455d-8edf-c40705723223 None None] MFA Rules not processed for user `b4d1899602114316a2f0d6364e5e2876`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:15.585721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-05a7252b-82e1-455d-8edf-c40705723223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:15.692290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1386/2723] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:15.704974 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b505a-70e3-482c-814c-5236abe05565 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:15.706983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-af1b505a-70e3-482c-814c-5236abe05565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:15.787805 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b505a-70e3-482c-814c-5236abe05565 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:15.794557 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64289e19-9c15-4716-af93-9aaf9659b41a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:15.795595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64289e19-9c15-4716-af93-9aaf9659b41a service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:15.796081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64289e19-9c15-4716-af93-9aaf9659b41a service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:15.796571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64289e19-9c15-4716-af93-9aaf9659b41a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:15.819033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64289e19-9c15-4716-af93-9aaf9659b41a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:15.819033 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64289e19-9c15-4716-af93-9aaf9659b41a service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:15.834057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1338/2724] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:40:25.686497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:25.827598 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=60,keystone:INSERT=3,keystone:DELETE=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:27.867320 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7252b-82e1-455d-8edf-c40705723223 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:27.981606 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7252b-82e1-455d-8edf-c40705723223 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:27.990281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:27.991505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:27.992015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:27.992527 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] PATH_INFO: `/v3/users/b4d1899602114316a2f0d6364e5e2876` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:27.995870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d1899602114316a2f0d6364e5e2876)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:27.999769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:28.295448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d1899602114316a2f0d6364e5e2876'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:28.296169 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d1899602114316a2f0d6364e5e2876'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:28.296806 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d1899602114316a2f0d6364e5e2876'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:28.415010 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d1899602114316a2f0d6364e5e2876'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:28.425877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d1899602114316a2f0d6364e5e2876'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:28.450215 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1387/2725] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:40:27 2022] DELETE /identity/v3/users/b4d1899602114316a2f0d6364e5e2876 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:28.455197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3fe6-5c9e-49fd-a04e-7a4192fb855b None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:28.456022 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3fe6-5c9e-49fd-a04e-7a4192fb855b None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:28.456022 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3fe6-5c9e-49fd-a04e-7a4192fb855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:28.478424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-d98a3fe6-5c9e-49fd-a04e-7a4192fb855b None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:28.484506 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d98a3fe6-5c9e-49fd-a04e-7a4192fb855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:28.551248 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1339/2726] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:28.563530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:28.565160 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:28.642531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b9f8e-9a44-4673-ad11-170cbc21d136 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:28.646256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaa27e-916a-4e80-a3c9-53da363c9bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:28.646925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaa27e-916a-4e80-a3c9-53da363c9bfa service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:28.647195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaa27e-916a-4e80-a3c9-53da363c9bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:28.647461 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaa27e-916a-4e80-a3c9-53da363c9bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:28.657989 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aaa27e-916a-4e80-a3c9-53da363c9bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:28.659474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aaa27e-916a-4e80-a3c9-53da363c9bfa service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:28.669826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1388/2727] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:40:29.255094 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a3fe6-5c9e-49fd-a04e-7a4192fb855b None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:29.256938 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d98a3fe6-5c9e-49fd-a04e-7a4192fb855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:29.270601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a3fe6-5c9e-49fd-a04e-7a4192fb855b None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:29.277607 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:29.278421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:29.278799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:29.279184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] PATH_INFO: `/v3/projects/7e30fc6bdbbb43bdb7fe777a35342c03` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:29.280706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e30fc6bdbbb43bdb7fe777a35342c03)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:29.282682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:29.451781 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e30fc6bdbbb43bdb7fe777a35342c03', 'request_id': 'req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:29.459234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e30fc6bdbbb43bdb7fe777a35342c03', 'request_id': 'req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:29.552731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e30fc6bdbbb43bdb7fe777a35342c03', 'request_id': 'req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:29.565746 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] The token cache is being invalidate because project 7e30fc6bdbbb43bdb7fe777a35342c03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:40:29.571876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:29.571876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-294e8246-ffbe-4ca3-8c5c-679b6dc1ed8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:29.571876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1340/2728] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:40:29 2022] DELETE /identity/v3/projects/7e30fc6bdbbb43bdb7fe777a35342c03 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:29.574272 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30bbe1-4324-4bae-a6ee-96a4d1d29afe None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:29.574833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30bbe1-4324-4bae-a6ee-96a4d1d29afe None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:29.574833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30bbe1-4324-4bae-a6ee-96a4d1d29afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:29.596447 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-9c30bbe1-4324-4bae-a6ee-96a4d1d29afe None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:29.598125 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9c30bbe1-4324-4bae-a6ee-96a4d1d29afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:29.620490 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:29.669419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1389/2729] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:29.746175 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce2cc3-05e3-486d-8685-bad0ff7a96e6 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:29.746595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce2cc3-05e3-486d-8685-bad0ff7a96e6 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:29.746879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce2cc3-05e3-486d-8685-bad0ff7a96e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:29.781071 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-b0ce2cc3-05e3-486d-8685-bad0ff7a96e6 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:29.782329 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b0ce2cc3-05e3-486d-8685-bad0ff7a96e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:29.791843 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1341/2730] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:29.796424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30bbe1-4324-4bae-a6ee-96a4d1d29afe None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:29.809305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30bbe1-4324-4bae-a6ee-96a4d1d29afe None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:29.816694 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34af71-f85c-4526-89e4-cc4221f659fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:29.817760 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34af71-f85c-4526-89e4-cc4221f659fe admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:29.818253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34af71-f85c-4526-89e4-cc4221f659fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:29.818929 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34af71-f85c-4526-89e4-cc4221f659fe admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:29.820118 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34af71-f85c-4526-89e4-cc4221f659fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:40:29.820612 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34af71-f85c-4526-89e4-cc4221f659fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:29.824198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34af71-f85c-4526-89e4-cc4221f659fe admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:29.845645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1390/2731] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:40:29.850632 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce2cc3-05e3-486d-8685-bad0ff7a96e6 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:29.864113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce2cc3-05e3-486d-8685-bad0ff7a96e6 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:29.870637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a85c2b-02cb-4845-9424-29f72e645bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:29.871383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a85c2b-02cb-4845-9424-29f72e645bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:29.871755 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a85c2b-02cb-4845-9424-29f72e645bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:29.872127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a85c2b-02cb-4845-9424-29f72e645bb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:29.872880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a85c2b-02cb-4845-9424-29f72e645bb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:29.875928 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a85c2b-02cb-4845-9424-29f72e645bb9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:30.047504 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1342/2732] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:40:29 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:40:30.049429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34af71-f85c-4526-89e4-cc4221f659fe admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:30.060858 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34af71-f85c-4526-89e4-cc4221f659fe admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:30.137019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151d222-9604-4a47-a85c-acd121496c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:30.137649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151d222-9604-4a47-a85c-acd121496c7f admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:30.137923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151d222-9604-4a47-a85c-acd121496c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:30.138192 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151d222-9604-4a47-a85c-acd121496c7f admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:30.138869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151d222-9604-4a47-a85c-acd121496c7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:30.140932 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151d222-9604-4a47-a85c-acd121496c7f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:30.297036 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1391/2733] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:40:30 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:40:30.302224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a85c2b-02cb-4845-9424-29f72e645bb9 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:30.317826 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a85c2b-02cb-4845-9424-29f72e645bb9 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:30.326017 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c7ec2-cc5d-4e9b-af13-7fce25878ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:30.326959 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c7ec2-cc5d-4e9b-af13-7fce25878ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:30.327345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c7ec2-cc5d-4e9b-af13-7fce25878ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:30.327766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c7ec2-cc5d-4e9b-af13-7fce25878ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:30.328896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c7ec2-cc5d-4e9b-af13-7fce25878ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:30.331853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c7ec2-cc5d-4e9b-af13-7fce25878ab7 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:30.344955 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1343/2734] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:30.349664 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151d222-9604-4a47-a85c-acd121496c7f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:30.361305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151d222-9604-4a47-a85c-acd121496c7f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:30.366137 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28453ecf-c95b-4b53-8037-72336dd08540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:30.366853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28453ecf-c95b-4b53-8037-72336dd08540 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:30.367132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28453ecf-c95b-4b53-8037-72336dd08540 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:30.367413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28453ecf-c95b-4b53-8037-72336dd08540 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:30.368107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28453ecf-c95b-4b53-8037-72336dd08540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:30.370330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28453ecf-c95b-4b53-8037-72336dd08540 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:30.383396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1392/2735] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:30.388626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c7ec2-cc5d-4e9b-af13-7fce25878ab7 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:30.400659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c7ec2-cc5d-4e9b-af13-7fce25878ab7 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:30.406012 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2323fb8-cb52-4a42-8371-3d34b19af6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:30.406831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2323fb8-cb52-4a42-8371-3d34b19af6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:30.407210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2323fb8-cb52-4a42-8371-3d34b19af6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:30.407645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2323fb8-cb52-4a42-8371-3d34b19af6a5 admin admin] PATH_INFO: `/v3/projects/8729a9fbbe8d48789eaaffc65a956d77/users/6b10b86e18ba49858a57206f8581af35/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:30.429121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2323fb8-cb52-4a42-8371-3d34b19af6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8729a9fbbe8d48789eaaffc65a956d77, user_id=6b10b86e18ba49858a57206f8581af35, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:30.435431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2323fb8-cb52-4a42-8371-3d34b19af6a5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:30.531600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1344/2736] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:40:30 2022] PUT /identity/v3/projects/8729a9fbbe8d48789eaaffc65a956d77/users/6b10b86e18ba49858a57206f8581af35/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:30.542944 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6da712-57b5-40b5-b193-3762604dd5ae None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:30.544233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6da712-57b5-40b5-b193-3762604dd5ae None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:30.544233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6da712-57b5-40b5-b193-3762604dd5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:30.584906 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-7f6da712-57b5-40b5-b193-3762604dd5ae None None] MFA Rules not processed for user `6b10b86e18ba49858a57206f8581af35`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:30.587788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7f6da712-57b5-40b5-b193-3762604dd5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:30.675020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1393/2737] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:30.679829 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2323fb8-cb52-4a42-8371-3d34b19af6a5 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:30.703958 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:30.755656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2323fb8-cb52-4a42-8371-3d34b19af6a5 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:30.763795 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b7dca-cafb-4362-998a-3fd44fab486b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:30.764685 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b7dca-cafb-4362-998a-3fd44fab486b admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:30.765136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b7dca-cafb-4362-998a-3fd44fab486b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:30.765552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b7dca-cafb-4362-998a-3fd44fab486b admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:30.766472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b7dca-cafb-4362-998a-3fd44fab486b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:30.769873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b7dca-cafb-4362-998a-3fd44fab486b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:30.887255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1345/2738] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:40:30 2022] POST /identity/v3/projects => generated 377 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:40:30.891640 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6da712-57b5-40b5-b193-3762604dd5ae None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:30.904382 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6da712-57b5-40b5-b193-3762604dd5ae None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:30.919795 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:30.968331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c9983-e51e-460e-a9b1-df766d7c5bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:30.969112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c9983-e51e-460e-a9b1-df766d7c5bfe admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:30.969430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c9983-e51e-460e-a9b1-df766d7c5bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:30.969755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c9983-e51e-460e-a9b1-df766d7c5bfe admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:30.970507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c9983-e51e-460e-a9b1-df766d7c5bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:30.972575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c9983-e51e-460e-a9b1-df766d7c5bfe admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:31.074545 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1394/2739] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:40:30 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:40:31.079216 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b7dca-cafb-4362-998a-3fd44fab486b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:31.091366 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b7dca-cafb-4362-998a-3fd44fab486b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:31.096345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2addb7-6c5b-4aca-942b-16edafe8bd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:31.096972 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2addb7-6c5b-4aca-942b-16edafe8bd87 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:31.097250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2addb7-6c5b-4aca-942b-16edafe8bd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:31.097523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2addb7-6c5b-4aca-942b-16edafe8bd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:31.098180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2addb7-6c5b-4aca-942b-16edafe8bd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:31.099982 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2addb7-6c5b-4aca-942b-16edafe8bd87 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:31.112355 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1346/2740] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:31.116996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c9983-e51e-460e-a9b1-df766d7c5bfe admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:31.128611 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c9983-e51e-460e-a9b1-df766d7c5bfe admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:31.133362 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942f3bb-306e-4e61-9361-8bca91948cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:31.134598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f3bb-306e-4e61-9361-8bca91948cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:31.134873 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f3bb-306e-4e61-9361-8bca91948cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:31.135144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f3bb-306e-4e61-9361-8bca91948cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:31.135915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942f3bb-306e-4e61-9361-8bca91948cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:31.137598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942f3bb-306e-4e61-9361-8bca91948cf6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:31.150644 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1395/2741] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:31.155078 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2addb7-6c5b-4aca-942b-16edafe8bd87 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:31.167070 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2addb7-6c5b-4aca-942b-16edafe8bd87 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:31.172079 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cd691-4b1a-4a80-883f-5551cb6b5612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:31.172750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cd691-4b1a-4a80-883f-5551cb6b5612 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:31.173047 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cd691-4b1a-4a80-883f-5551cb6b5612 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:31.173316 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cd691-4b1a-4a80-883f-5551cb6b5612 admin admin] PATH_INFO: `/v3/projects/fa6f88ec5e0346119ae67235798e54a1/users/5d8f4c6a652a462e9f9b68d44d28d3e3/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:31.190263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cd691-4b1a-4a80-883f-5551cb6b5612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6f88ec5e0346119ae67235798e54a1, user_id=5d8f4c6a652a462e9f9b68d44d28d3e3, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:31.193065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cd691-4b1a-4a80-883f-5551cb6b5612 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:31.297437 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1347/2742] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:40:31 2022] PUT /identity/v3/projects/fa6f88ec5e0346119ae67235798e54a1/users/5d8f4c6a652a462e9f9b68d44d28d3e3/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:31.309336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356bef9-3b90-4ef0-bd9d-b3e4df1692ec None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:31.309667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356bef9-3b90-4ef0-bd9d-b3e4df1692ec None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:31.309996 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356bef9-3b90-4ef0-bd9d-b3e4df1692ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:31.346406 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-7356bef9-3b90-4ef0-bd9d-b3e4df1692ec None None] MFA Rules not processed for user `5d8f4c6a652a462e9f9b68d44d28d3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:31.347660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7356bef9-3b90-4ef0-bd9d-b3e4df1692ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:31.423748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1396/2743] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:31.428721 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cd691-4b1a-4a80-883f-5551cb6b5612 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:31.500523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cd691-4b1a-4a80-883f-5551cb6b5612 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:31.505292 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e8846-07c9-4434-9bc1-55aa9cb45ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:31.505920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e8846-07c9-4434-9bc1-55aa9cb45ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:31.506187 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e8846-07c9-4434-9bc1-55aa9cb45ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:31.506454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e8846-07c9-4434-9bc1-55aa9cb45ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:31.507112 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e8846-07c9-4434-9bc1-55aa9cb45ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:31.509799 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e8846-07c9-4434-9bc1-55aa9cb45ca6 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:31.603678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1348/2744] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:40:31 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:40:31.604943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356bef9-3b90-4ef0-bd9d-b3e4df1692ec None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:31.628234 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356bef9-3b90-4ef0-bd9d-b3e4df1692ec None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:31.700753 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae70a2-d085-46f0-bd2f-ae444065d4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:31.701268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae70a2-d085-46f0-bd2f-ae444065d4eb admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:31.701551 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae70a2-d085-46f0-bd2f-ae444065d4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:31.701819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae70a2-d085-46f0-bd2f-ae444065d4eb admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:31.702521 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae70a2-d085-46f0-bd2f-ae444065d4eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:31.704465 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae70a2-d085-46f0-bd2f-ae444065d4eb admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:31.807440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1397/2745] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:40:31 2022] POST /identity/v3/users => generated 295 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:40:31.811461 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e8846-07c9-4434-9bc1-55aa9cb45ca6 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:31.824898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e8846-07c9-4434-9bc1-55aa9cb45ca6 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:31.830979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc73442-8a78-413b-8480-b927989638e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:31.831628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc73442-8a78-413b-8480-b927989638e3 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:31.836045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc73442-8a78-413b-8480-b927989638e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:31.836045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc73442-8a78-413b-8480-b927989638e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:31.836045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc73442-8a78-413b-8480-b927989638e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:31.836045 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc73442-8a78-413b-8480-b927989638e3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:31.849777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1349/2746] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:31.855248 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae70a2-d085-46f0-bd2f-ae444065d4eb admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:31.874449 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae70a2-d085-46f0-bd2f-ae444065d4eb admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:31.883549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f86948-b6b6-4feb-a6a8-4f7f8a43c339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:31.885953 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f86948-b6b6-4feb-a6a8-4f7f8a43c339 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:31.886469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f86948-b6b6-4feb-a6a8-4f7f8a43c339 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:31.887006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f86948-b6b6-4feb-a6a8-4f7f8a43c339 admin admin] PATH_INFO: `/v3/projects/0e3c153d4f5b44f5bc49a607d5e48b94/users/5154a481943f4cd081ca93474d7e9ca9/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:31.911904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:31.919032 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f86948-b6b6-4feb-a6a8-4f7f8a43c339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3c153d4f5b44f5bc49a607d5e48b94, user_id=5154a481943f4cd081ca93474d7e9ca9, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:31.921756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f86948-b6b6-4feb-a6a8-4f7f8a43c339 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:32.011445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1398/2747] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:40:31 2022] PUT /identity/v3/projects/0e3c153d4f5b44f5bc49a607d5e48b94/users/5154a481943f4cd081ca93474d7e9ca9/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:40:32.015526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc73442-8a78-413b-8480-b927989638e3 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:32.026732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc73442-8a78-413b-8480-b927989638e3 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:32.077946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:32.093800 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be07b1-9535-420b-9c66-bfca464b41e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:32.094411 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be07b1-9535-420b-9c66-bfca464b41e5 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:32.094722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be07b1-9535-420b-9c66-bfca464b41e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:32.094994 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be07b1-9535-420b-9c66-bfca464b41e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:32.095633 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be07b1-9535-420b-9c66-bfca464b41e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:32.097257 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be07b1-9535-420b-9c66-bfca464b41e5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:32.108442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1350/2748] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:32.112816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f86948-b6b6-4feb-a6a8-4f7f8a43c339 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:32.124604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f86948-b6b6-4feb-a6a8-4f7f8a43c339 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:32.131632 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca5f9a-9cd4-47af-b230-12d51bf321cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:32.132376 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca5f9a-9cd4-47af-b230-12d51bf321cd admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:32.132655 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca5f9a-9cd4-47af-b230-12d51bf321cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:32.132948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca5f9a-9cd4-47af-b230-12d51bf321cd admin admin] PATH_INFO: `/v3/projects/0e3c153d4f5b44f5bc49a607d5e48b94/users/5154a481943f4cd081ca93474d7e9ca9/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:32.135357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca5f9a-9cd4-47af-b230-12d51bf321cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3c153d4f5b44f5bc49a607d5e48b94, user_id=5154a481943f4cd081ca93474d7e9ca9, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:32.138684 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca5f9a-9cd4-47af-b230-12d51bf321cd admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:32.150204 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1399/2749] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:40:32 2022] PUT /identity/v3/projects/0e3c153d4f5b44f5bc49a607d5e48b94/users/5154a481943f4cd081ca93474d7e9ca9/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:32.161880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99ab50-752c-4354-b89e-f5fd7fee7762 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:32.162505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99ab50-752c-4354-b89e-f5fd7fee7762 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:32.162984 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99ab50-752c-4354-b89e-f5fd7fee7762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:32.193426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-5e99ab50-752c-4354-b89e-f5fd7fee7762 None None] MFA Rules not processed for user `5154a481943f4cd081ca93474d7e9ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:32.194690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5e99ab50-752c-4354-b89e-f5fd7fee7762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:32.276091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1351/2750] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:32.287822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca5f9a-9cd4-47af-b230-12d51bf321cd admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:32.289639 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-88ca5f9a-9cd4-47af-b230-12d51bf321cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:32.363480 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca5f9a-9cd4-47af-b230-12d51bf321cd admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:32.367334 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ba8ef-1b04-40ae-ae7d-c1cd64b842bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:32.368186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ba8ef-1b04-40ae-ae7d-c1cd64b842bf service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:32.368575 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ba8ef-1b04-40ae-ae7d-c1cd64b842bf service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:32.368951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ba8ef-1b04-40ae-ae7d-c1cd64b842bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:32.457191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ba8ef-1b04-40ae-ae7d-c1cd64b842bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:32.459503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ba8ef-1b04-40ae-ae7d-c1cd64b842bf service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:32.472053 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1400/2751] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:40:34.553522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99ab50-752c-4354-b89e-f5fd7fee7762 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:34.556166 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5e99ab50-752c-4354-b89e-f5fd7fee7762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:34.576523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99ab50-752c-4354-b89e-f5fd7fee7762 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:34.583863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201277e5-fcf6-4b7c-a626-cdce626ec59c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:34.584507 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201277e5-fcf6-4b7c-a626-cdce626ec59c service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:34.584929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201277e5-fcf6-4b7c-a626-cdce626ec59c service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:34.585346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201277e5-fcf6-4b7c-a626-cdce626ec59c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:34.685035 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201277e5-fcf6-4b7c-a626-cdce626ec59c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:34.688217 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201277e5-fcf6-4b7c-a626-cdce626ec59c service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:34.707197 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1352/2752] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:40:34.732961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ba8ef-1b04-40ae-ae7d-c1cd64b842bf service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:34.745184 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ba8ef-1b04-40ae-ae7d-c1cd64b842bf service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:34.749485 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0f91f-3f7c-4532-915b-83f4198210da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:34.750467 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f91f-3f7c-4532-915b-83f4198210da service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:34.750870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f91f-3f7c-4532-915b-83f4198210da service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:34.751542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f91f-3f7c-4532-915b-83f4198210da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:34.753882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-44f0f91f-3f7c-4532-915b-83f4198210da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:34.857903 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0f91f-3f7c-4532-915b-83f4198210da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:34.860540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0f91f-3f7c-4532-915b-83f4198210da service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:34.874244 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1401/2753] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:40:36.635452 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201277e5-fcf6-4b7c-a626-cdce626ec59c service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:36.646636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201277e5-fcf6-4b7c-a626-cdce626ec59c service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:36.651911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe170d3-60cd-49ca-8e6d-bdffd5df9c93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:36.652623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe170d3-60cd-49ca-8e6d-bdffd5df9c93 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:36.652943 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe170d3-60cd-49ca-8e6d-bdffd5df9c93 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:36.653261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe170d3-60cd-49ca-8e6d-bdffd5df9c93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:36.664569 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe170d3-60cd-49ca-8e6d-bdffd5df9c93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:36.666523 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe170d3-60cd-49ca-8e6d-bdffd5df9c93 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:36.678021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1353/2754] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:40:44.868217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:46.672864 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=77 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:47.916173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac5931-e620-4a3f-9188-fbe2749207ee None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:47.917116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac5931-e620-4a3f-9188-fbe2749207ee None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:47.917116 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac5931-e620-4a3f-9188-fbe2749207ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:47.940415 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f1ac5931-e620-4a3f-9188-fbe2749207ee None None] MFA Rules not processed for user `aa55b9fade0047c68c2f3f6bf4dfe44e`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:47.946178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f1ac5931-e620-4a3f-9188-fbe2749207ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:47.950281 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1402/2755] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:50.405180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe170d3-60cd-49ca-8e6d-bdffd5df9c93 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:50.477950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe170d3-60cd-49ca-8e6d-bdffd5df9c93 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:50.483136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:50.483801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:50.484082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:50.484356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] PATH_INFO: `/v3/users/6b10b86e18ba49858a57206f8581af35` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:50.485636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b10b86e18ba49858a57206f8581af35)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:50.487736 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:50.728717 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b10b86e18ba49858a57206f8581af35'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:50.729425 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b10b86e18ba49858a57206f8581af35'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:50.730055 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b10b86e18ba49858a57206f8581af35'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:50.737767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b10b86e18ba49858a57206f8581af35'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:50.752686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b10b86e18ba49858a57206f8581af35'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:50.841699 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1354/2756] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:40:50 2022] DELETE /identity/v3/users/6b10b86e18ba49858a57206f8581af35 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:50.846557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ca3a0-26fe-448c-bf77-2dbf3deccb83 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:50.847372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ca3a0-26fe-448c-bf77-2dbf3deccb83 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:50.847559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ca3a0-26fe-448c-bf77-2dbf3deccb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:50.869668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-bd5ca3a0-26fe-448c-bf77-2dbf3deccb83 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:50.871089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bd5ca3a0-26fe-448c-bf77-2dbf3deccb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:50.939781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1403/2757] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:50.951726 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:50.954380 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:51.033323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16510e4e-88d6-49c3-8c45-25ea5656b9fa admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:51.038137 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a1023-ab7f-4eb1-adbc-f05d03a810fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:51.041185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a1023-ab7f-4eb1-adbc-f05d03a810fa service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:51.041185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a1023-ab7f-4eb1-adbc-f05d03a810fa service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:51.041185 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a1023-ab7f-4eb1-adbc-f05d03a810fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:51.051354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a1023-ab7f-4eb1-adbc-f05d03a810fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:51.052887 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a1023-ab7f-4eb1-adbc-f05d03a810fa service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:51.066330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1355/2758] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:40:51.719900 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ca3a0-26fe-448c-bf77-2dbf3deccb83 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:51.722719 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bd5ca3a0-26fe-448c-bf77-2dbf3deccb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:51.748444 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ca3a0-26fe-448c-bf77-2dbf3deccb83 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:51.759877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:51.760954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:51.761540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:51.761894 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] PATH_INFO: `/v3/projects/8729a9fbbe8d48789eaaffc65a956d77` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:51.764132 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8729a9fbbe8d48789eaaffc65a956d77)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:51.768086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:51.951153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8729a9fbbe8d48789eaaffc65a956d77', 'request_id': 'req-8bc4e416-7db8-47b8-a5a4-c01b700b3825'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:51.958518 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8729a9fbbe8d48789eaaffc65a956d77', 'request_id': 'req-8bc4e416-7db8-47b8-a5a4-c01b700b3825'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:51.966865 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8729a9fbbe8d48789eaaffc65a956d77', 'request_id': 'req-8bc4e416-7db8-47b8-a5a4-c01b700b3825'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:52.101422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] The token cache is being invalidate because project 8729a9fbbe8d48789eaaffc65a956d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:40:52.102904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:52.104005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:52.106194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1404/2759] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:40:51 2022] DELETE /identity/v3/projects/8729a9fbbe8d48789eaaffc65a956d77 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:52.110210 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a1023-ab7f-4eb1-adbc-f05d03a810fa service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:52.125890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a1023-ab7f-4eb1-adbc-f05d03a810fa service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:52.152292 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:52.217129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:52.218250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:52.218794 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:52.219280 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] PATH_INFO: `/v3/users/5d8f4c6a652a462e9f9b68d44d28d3e3` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:52.220970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d8f4c6a652a462e9f9b68d44d28d3e3)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:52.224092 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:52.510605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d8f4c6a652a462e9f9b68d44d28d3e3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:52.511730 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d8f4c6a652a462e9f9b68d44d28d3e3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:52.512709 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d8f4c6a652a462e9f9b68d44d28d3e3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:52.529820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d8f4c6a652a462e9f9b68d44d28d3e3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:52.556528 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d8f4c6a652a462e9f9b68d44d28d3e3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:52.731571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1356/2760] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:40:52 2022] DELETE /identity/v3/users/5d8f4c6a652a462e9f9b68d44d28d3e3 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:40:53.757093 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:53.759203 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:53.782278 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e416-7db8-47b8-a5a4-c01b700b3825 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:53.854181 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:53.855086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:53.855375 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:53.855652 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] PATH_INFO: `/v3/projects/fa6f88ec5e0346119ae67235798e54a1` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:53.856943 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa6f88ec5e0346119ae67235798e54a1)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:53.859180 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:53.886052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:54.104634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa6f88ec5e0346119ae67235798e54a1', 'request_id': 'req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:54.118953 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa6f88ec5e0346119ae67235798e54a1', 'request_id': 'req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:54.139577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa6f88ec5e0346119ae67235798e54a1', 'request_id': 'req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:54.240540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] The token cache is being invalidate because project fa6f88ec5e0346119ae67235798e54a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:40:54.240885 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:54.241651 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:54.244568 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1405/2761] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:40:53 2022] DELETE /identity/v3/projects/fa6f88ec5e0346119ae67235798e54a1 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:54.248997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:54.254827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:54.267917 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196610aa-f137-4e90-80b8-4d02a321cdd4 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:54.356557 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:54.357630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:54.358117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:54.358704 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] PATH_INFO: `/v3/users/5154a481943f4cd081ca93474d7e9ca9` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:54.361400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5154a481943f4cd081ca93474d7e9ca9)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:54.365650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:54.634567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:54.639468 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5154a481943f4cd081ca93474d7e9ca9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:54.641432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5154a481943f4cd081ca93474d7e9ca9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:54.641550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5154a481943f4cd081ca93474d7e9ca9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:54.654156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5154a481943f4cd081ca93474d7e9ca9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:54.672420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-85bf2387-81ce-4c60-aa63-585aab4401f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5154a481943f4cd081ca93474d7e9ca9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:54.781661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1357/2762] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:40:54 2022] DELETE /identity/v3/users/5154a481943f4cd081ca93474d7e9ca9 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:40:55.441920 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:55.445542 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:55.470452 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5e45e-a4e2-4efe-8c23-44fcf3039270 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:55.556633 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:55.557395 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:55.557668 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:55.557952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] PATH_INFO: `/v3/projects/0e3c153d4f5b44f5bc49a607d5e48b94` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:55.559481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e3c153d4f5b44f5bc49a607d5e48b94)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:55.561476 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:55.726413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e3c153d4f5b44f5bc49a607d5e48b94', 'request_id': 'req-75f69edf-86df-4a7e-8543-1a251a67e1c6'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:55.736129 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e3c153d4f5b44f5bc49a607d5e48b94', 'request_id': 'req-75f69edf-86df-4a7e-8543-1a251a67e1c6'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:55.746332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e3c153d4f5b44f5bc49a607d5e48b94', 'request_id': 'req-75f69edf-86df-4a7e-8543-1a251a67e1c6'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:55.917651 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] The token cache is being invalidate because project 0e3c153d4f5b44f5bc49a607d5e48b94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:40:55.938314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:55.938314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-75f69edf-86df-4a7e-8543-1a251a67e1c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:55.938314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1406/2763] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:40:55 2022] DELETE /identity/v3/projects/0e3c153d4f5b44f5bc49a607d5e48b94 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:40:55.939340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab96bd-a7e3-47ca-9267-6a8071d21b0d None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:55.939340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab96bd-a7e3-47ca-9267-6a8071d21b0d None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:55.939340 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab96bd-a7e3-47ca-9267-6a8071d21b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:55.949193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-0aab96bd-a7e3-47ca-9267-6a8071d21b0d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:55.950456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0aab96bd-a7e3-47ca-9267-6a8071d21b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:56.016449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1358/2764] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:56.026240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249b7a9-dc18-440c-ba4c-040dc5966d5f None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:56.026591 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249b7a9-dc18-440c-ba4c-040dc5966d5f None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:56.026818 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249b7a9-dc18-440c-ba4c-040dc5966d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:56.049565 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0249b7a9-dc18-440c-ba4c-040dc5966d5f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:56.050854 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0249b7a9-dc18-440c-ba4c-040dc5966d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:56.071142 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1407/2765] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:56.073946 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab96bd-a7e3-47ca-9267-6a8071d21b0d None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:56.084600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab96bd-a7e3-47ca-9267-6a8071d21b0d None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:56.090761 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263e779-a543-470f-ae65-28e9fa2f15f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:56.091392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263e779-a543-470f-ae65-28e9fa2f15f7 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:56.091677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263e779-a543-470f-ae65-28e9fa2f15f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:56.091933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263e779-a543-470f-ae65-28e9fa2f15f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:56.092592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263e779-a543-470f-ae65-28e9fa2f15f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:40:56.092890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263e779-a543-470f-ae65-28e9fa2f15f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:56.094552 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263e779-a543-470f-ae65-28e9fa2f15f7 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:56.107742 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1359/2766] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 17:40:56.112571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249b7a9-dc18-440c-ba4c-040dc5966d5f None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:56.123374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249b7a9-dc18-440c-ba4c-040dc5966d5f None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:56.128522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ffc42-a72f-4565-a18c-65548611b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:56.129153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ffc42-a72f-4565-a18c-65548611b43f admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:56.129433 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ffc42-a72f-4565-a18c-65548611b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:56.129702 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ffc42-a72f-4565-a18c-65548611b43f admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:56.130368 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ffc42-a72f-4565-a18c-65548611b43f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:56.132374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ffc42-a72f-4565-a18c-65548611b43f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:56.246374 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1408/2767] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:40:56 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:40:56.251526 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263e779-a543-470f-ae65-28e9fa2f15f7 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:56.263101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263e779-a543-470f-ae65-28e9fa2f15f7 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:56.330030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88aa34-6a7e-4ff5-9ea0-8d88b51db676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:56.330694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88aa34-6a7e-4ff5-9ea0-8d88b51db676 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:56.330967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88aa34-6a7e-4ff5-9ea0-8d88b51db676 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:56.331247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88aa34-6a7e-4ff5-9ea0-8d88b51db676 admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:56.331882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88aa34-6a7e-4ff5-9ea0-8d88b51db676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:56.333833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88aa34-6a7e-4ff5-9ea0-8d88b51db676 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:56.423183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1360/2768] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:40:56 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:40:56.427993 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ffc42-a72f-4565-a18c-65548611b43f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:56.439982 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ffc42-a72f-4565-a18c-65548611b43f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:56.444901 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11099b1a-2032-44b6-9e78-39f8346d4163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:56.446492 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11099b1a-2032-44b6-9e78-39f8346d4163 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:56.446839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11099b1a-2032-44b6-9e78-39f8346d4163 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:56.447112 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11099b1a-2032-44b6-9e78-39f8346d4163 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:56.447756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11099b1a-2032-44b6-9e78-39f8346d4163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:56.449353 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11099b1a-2032-44b6-9e78-39f8346d4163 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:56.451895 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:56.461549 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1409/2769] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 17:40:56.465970 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88aa34-6a7e-4ff5-9ea0-8d88b51db676 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:56.477625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88aa34-6a7e-4ff5-9ea0-8d88b51db676 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:56.482825 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c7a7a-f6da-4bcb-bffd-e4e8319bb046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:56.487807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c7a7a-f6da-4bcb-bffd-e4e8319bb046 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:56.487807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c7a7a-f6da-4bcb-bffd-e4e8319bb046 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:56.487807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c7a7a-f6da-4bcb-bffd-e4e8319bb046 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:56.487807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c7a7a-f6da-4bcb-bffd-e4e8319bb046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:56.487807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c7a7a-f6da-4bcb-bffd-e4e8319bb046 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:56.505448 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1361/2770] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:56.511921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11099b1a-2032-44b6-9e78-39f8346d4163 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:56.532677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11099b1a-2032-44b6-9e78-39f8346d4163 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:56.542522 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa2462-4f9b-499a-9358-29c79fbe3be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:56.545337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa2462-4f9b-499a-9358-29c79fbe3be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:56.545337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa2462-4f9b-499a-9358-29c79fbe3be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:56.545337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa2462-4f9b-499a-9358-29c79fbe3be7 admin admin] PATH_INFO: `/v3/projects/91d32762c19340eca3895ec3b47647f2/users/0c2c4da147354d58ab7bec26b0e36cb9/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:56.573758 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa2462-4f9b-499a-9358-29c79fbe3be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d32762c19340eca3895ec3b47647f2, user_id=0c2c4da147354d58ab7bec26b0e36cb9, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:56.576592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa2462-4f9b-499a-9358-29c79fbe3be7 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:56.696370 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1410/2771] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:40:56 2022] PUT /identity/v3/projects/91d32762c19340eca3895ec3b47647f2/users/0c2c4da147354d58ab7bec26b0e36cb9/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:40:56.717356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14973000-4f4a-4b4f-9116-40a75453796a None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:56.717671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14973000-4f4a-4b4f-9116-40a75453796a None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:56.717903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14973000-4f4a-4b4f-9116-40a75453796a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:56.752084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-14973000-4f4a-4b4f-9116-40a75453796a None None] MFA Rules not processed for user `0c2c4da147354d58ab7bec26b0e36cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:56.753360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-14973000-4f4a-4b4f-9116-40a75453796a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:56.771322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:56.831014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1362/2772] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:56.843091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa2462-4f9b-499a-9358-29c79fbe3be7 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:56.844870 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-19aa2462-4f9b-499a-9358-29c79fbe3be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:56.913012 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa2462-4f9b-499a-9358-29c79fbe3be7 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:56.917796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3c7ec-ff66-4ba9-8267-854ad4847ac2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:56.918427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3c7ec-ff66-4ba9-8267-854ad4847ac2 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:56.918737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3c7ec-ff66-4ba9-8267-854ad4847ac2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:56.919017 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3c7ec-ff66-4ba9-8267-854ad4847ac2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:56.929305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3c7ec-ff66-4ba9-8267-854ad4847ac2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:56.931590 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3c7ec-ff66-4ba9-8267-854ad4847ac2 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:56.942177 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1411/2773] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:40:56.968816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14973000-4f4a-4b4f-9116-40a75453796a None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:57.040027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14973000-4f4a-4b4f-9116-40a75453796a None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:57.044615 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:57.045254 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:57.045528 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:57.045806 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] PATH_INFO: `/v3/users/0c2c4da147354d58ab7bec26b0e36cb9` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:57.046977 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c2c4da147354d58ab7bec26b0e36cb9)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:57.048953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:57.343302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c2c4da147354d58ab7bec26b0e36cb9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:57.344390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c2c4da147354d58ab7bec26b0e36cb9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:57.345372 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c2c4da147354d58ab7bec26b0e36cb9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:57.358393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c2c4da147354d58ab7bec26b0e36cb9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:57.374472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c2c4da147354d58ab7bec26b0e36cb9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:57.459953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1363/2774] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:40:56 2022] DELETE /identity/v3/users/0c2c4da147354d58ab7bec26b0e36cb9 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:57.467361 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7a3d7-c4de-4d32-b286-569caf659bf4 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:57.467946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7a3d7-c4de-4d32-b286-569caf659bf4 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:57.468068 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7a3d7-c4de-4d32-b286-569caf659bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:57.490246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-50f7a3d7-c4de-4d32-b286-569caf659bf4 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:57.491501 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-50f7a3d7-c4de-4d32-b286-569caf659bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:57.576390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1412/2775] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:57.588531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:57.590694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:57.637216 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:57.655694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2da366-15c4-45e2-bf97-72a0a26e9df7 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:57.659609 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7c96b-9721-419e-865b-56bdf76a68ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:57.660236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7c96b-9721-419e-865b-56bdf76a68ca service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:57.660513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7c96b-9721-419e-865b-56bdf76a68ca service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:57.660802 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7c96b-9721-419e-865b-56bdf76a68ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:57.673472 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7c96b-9721-419e-865b-56bdf76a68ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:57.675637 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7c96b-9721-419e-865b-56bdf76a68ca service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:57.689451 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1364/2776] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:40:58.147976 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7a3d7-c4de-4d32-b286-569caf659bf4 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:58.150671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-50f7a3d7-c4de-4d32-b286-569caf659bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:58.167377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7a3d7-c4de-4d32-b286-569caf659bf4 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:58.176091 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:58.177071 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:58.177556 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:58.178033 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] PATH_INFO: `/v3/projects/91d32762c19340eca3895ec3b47647f2` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:58.180904 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d32762c19340eca3895ec3b47647f2)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:58.184690 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:58.297821 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d32762c19340eca3895ec3b47647f2', 'request_id': 'req-89342f37-7677-471d-86fd-adfcc59b99eb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:58.305763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d32762c19340eca3895ec3b47647f2', 'request_id': 'req-89342f37-7677-471d-86fd-adfcc59b99eb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:58.313402 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d32762c19340eca3895ec3b47647f2', 'request_id': 'req-89342f37-7677-471d-86fd-adfcc59b99eb'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:58.400085 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:58.406306 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] The token cache is being invalidate because project 91d32762c19340eca3895ec3b47647f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:40:58.406613 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:58.407315 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-89342f37-7677-471d-86fd-adfcc59b99eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:40:58.408841 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1413/2777] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:40:58 2022] DELETE /identity/v3/projects/91d32762c19340eca3895ec3b47647f2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:58.413836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d321b8-6439-47a9-8047-63ad88b8cd8c None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:58.414618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d321b8-6439-47a9-8047-63ad88b8cd8c None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:58.414967 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d321b8-6439-47a9-8047-63ad88b8cd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:58.437524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-f5d321b8-6439-47a9-8047-63ad88b8cd8c None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:58.438952 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f5d321b8-6439-47a9-8047-63ad88b8cd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:58.528745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1365/2778] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:40:58.536945 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb861031-0a00-43b5-bd14-442211e1523d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:58.537500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb861031-0a00-43b5-bd14-442211e1523d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:58.537500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb861031-0a00-43b5-bd14-442211e1523d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:58.567563 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-bb861031-0a00-43b5-bd14-442211e1523d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:58.568805 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bb861031-0a00-43b5-bd14-442211e1523d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:58.578014 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1414/2779] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:58.583019 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d321b8-6439-47a9-8047-63ad88b8cd8c None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:58.598935 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d321b8-6439-47a9-8047-63ad88b8cd8c None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:58.605401 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555223d-f316-4c76-8854-a37f6ef6273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:58.606234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555223d-f316-4c76-8854-a37f6ef6273d admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:58.606644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555223d-f316-4c76-8854-a37f6ef6273d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:58.607114 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555223d-f316-4c76-8854-a37f6ef6273d admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:58.607893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555223d-f316-4c76-8854-a37f6ef6273d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:40:58.608286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555223d-f316-4c76-8854-a37f6ef6273d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:58.611080 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555223d-f316-4c76-8854-a37f6ef6273d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:58.635339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1366/2780] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:40:58.641341 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb861031-0a00-43b5-bd14-442211e1523d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:58.652666 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb861031-0a00-43b5-bd14-442211e1523d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:58.657559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ada26a-a3a8-42c4-b8ff-90c3ca53a379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:58.658182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ada26a-a3a8-42c4-b8ff-90c3ca53a379 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:58.658457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ada26a-a3a8-42c4-b8ff-90c3ca53a379 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:58.658764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ada26a-a3a8-42c4-b8ff-90c3ca53a379 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:58.659428 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ada26a-a3a8-42c4-b8ff-90c3ca53a379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:58.661426 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ada26a-a3a8-42c4-b8ff-90c3ca53a379 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:58.788788 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1415/2781] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:40:58 2022] POST /identity/v3/projects => generated 395 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:40:58.794144 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555223d-f316-4c76-8854-a37f6ef6273d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:58.805241 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555223d-f316-4c76-8854-a37f6ef6273d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:58.875762 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4164e49-c756-4964-82dc-88408bf2c70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:58.876677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4164e49-c756-4964-82dc-88408bf2c70d admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:58.877076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4164e49-c756-4964-82dc-88408bf2c70d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:58.877466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4164e49-c756-4964-82dc-88408bf2c70d admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:58.878428 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4164e49-c756-4964-82dc-88408bf2c70d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:58.881560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4164e49-c756-4964-82dc-88408bf2c70d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:58.991361 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1367/2782] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:40:58 2022] POST /identity/v3/users => generated 298 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:40:58.993534 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ada26a-a3a8-42c4-b8ff-90c3ca53a379 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:59.006367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ada26a-a3a8-42c4-b8ff-90c3ca53a379 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:59.012101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c78a6-14be-4829-910d-f042fe03a987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:59.012739 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c78a6-14be-4829-910d-f042fe03a987 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:59.013102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c78a6-14be-4829-910d-f042fe03a987 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:59.013298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c78a6-14be-4829-910d-f042fe03a987 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:59.013960 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c78a6-14be-4829-910d-f042fe03a987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:59.015633 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c78a6-14be-4829-910d-f042fe03a987 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:59.031552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1416/2783] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:59.035510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4164e49-c756-4964-82dc-88408bf2c70d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:59.046966 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4164e49-c756-4964-82dc-88408bf2c70d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:59.052181 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecee3b3-4b8a-4311-94f5-fd8c32654c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:59.052803 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee3b3-4b8a-4311-94f5-fd8c32654c49 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:59.053142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee3b3-4b8a-4311-94f5-fd8c32654c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:59.053417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee3b3-4b8a-4311-94f5-fd8c32654c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:59.054082 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecee3b3-4b8a-4311-94f5-fd8c32654c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:59.056345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecee3b3-4b8a-4311-94f5-fd8c32654c49 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:59.070661 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1368/2784] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:40:59.075266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c78a6-14be-4829-910d-f042fe03a987 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:59.087413 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c78a6-14be-4829-910d-f042fe03a987 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:59.094974 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702aac4-8f90-4c19-abab-d994b2dcc5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:59.096044 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702aac4-8f90-4c19-abab-d994b2dcc5f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:59.096422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702aac4-8f90-4c19-abab-d994b2dcc5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:59.096915 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702aac4-8f90-4c19-abab-d994b2dcc5f5 admin admin] PATH_INFO: `/v3/projects/90712f65960b439e92b1e73441412cfa/users/0e3f6d7474f14123aa603ba54248755b/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:59.121266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702aac4-8f90-4c19-abab-d994b2dcc5f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90712f65960b439e92b1e73441412cfa, user_id=0e3f6d7474f14123aa603ba54248755b, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:59.124520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702aac4-8f90-4c19-abab-d994b2dcc5f5 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:59.209584 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1417/2785] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:40:59 2022] PUT /identity/v3/projects/90712f65960b439e92b1e73441412cfa/users/0e3f6d7474f14123aa603ba54248755b/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:40:59.221281 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f9a93-d1f2-4469-be37-7ed68f8247d2 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:59.221563 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f9a93-d1f2-4469-be37-7ed68f8247d2 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:59.221839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f9a93-d1f2-4469-be37-7ed68f8247d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:59.252956 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-ce5f9a93-d1f2-4469-be37-7ed68f8247d2 None None] MFA Rules not processed for user `0e3f6d7474f14123aa603ba54248755b`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:40:59.254379 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ce5f9a93-d1f2-4469-be37-7ed68f8247d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:59.302998 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:40:59.339939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1369/2786] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:40:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:40:59.357380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702aac4-8f90-4c19-abab-d994b2dcc5f5 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:40:59.359999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9702aac4-8f90-4c19-abab-d994b2dcc5f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:40:59.440571 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702aac4-8f90-4c19-abab-d994b2dcc5f5 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:40:59.445425 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb3378-3ef7-4cc2-80b9-7270be2616e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:40:59.446337 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb3378-3ef7-4cc2-80b9-7270be2616e6 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:40:59.446771 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb3378-3ef7-4cc2-80b9-7270be2616e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:40:59.447055 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb3378-3ef7-4cc2-80b9-7270be2616e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:40:59.457731 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb3378-3ef7-4cc2-80b9-7270be2616e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:40:59.459261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb3378-3ef7-4cc2-80b9-7270be2616e6 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:40:59.471198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1418/2787] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:40:59 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:41:09.368061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=12 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:41:09.465736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:41:12.077957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f9a93-d1f2-4469-be37-7ed68f8247d2 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:41:12.150087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f9a93-d1f2-4469-be37-7ed68f8247d2 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:41:12.155142 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:41:12.155808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:41:12.156093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:41:12.156363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] PATH_INFO: `/v3/users/0e3f6d7474f14123aa603ba54248755b` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:41:12.157641 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e3f6d7474f14123aa603ba54248755b)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:41:12.159687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:41:12.431180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e3f6d7474f14123aa603ba54248755b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:41:12.431875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e3f6d7474f14123aa603ba54248755b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:41:12.432511 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e3f6d7474f14123aa603ba54248755b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:41:12.440536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e3f6d7474f14123aa603ba54248755b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:41:12.454497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e3f6d7474f14123aa603ba54248755b'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:41:12.563786 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1370/2788] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:41:12 2022] DELETE /identity/v3/users/0e3f6d7474f14123aa603ba54248755b => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:41:12.570046 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be7b0e-6803-409d-9c85-f98ed83832e5 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:41:12.570862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be7b0e-6803-409d-9c85-f98ed83832e5 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:41:12.570862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be7b0e-6803-409d-9c85-f98ed83832e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:41:12.595154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-17be7b0e-6803-409d-9c85-f98ed83832e5 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:41:12.596418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-17be7b0e-6803-409d-9c85-f98ed83832e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:41:12.660962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1419/2789] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:41:12.673093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:41:12.674774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:41:12.742831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b16d13-a2d0-4a39-a9c0-100b797970c5 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:41:12.746758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb229ebd-118d-44fa-9a33-14b0ff744c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:41:12.747382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229ebd-118d-44fa-9a33-14b0ff744c2d service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:41:12.747656 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229ebd-118d-44fa-9a33-14b0ff744c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:41:12.747925 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229ebd-118d-44fa-9a33-14b0ff744c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:41:12.758737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb229ebd-118d-44fa-9a33-14b0ff744c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:41:12.760204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb229ebd-118d-44fa-9a33-14b0ff744c2d service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:41:12.771348 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1371/2790] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:41:13.302253 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be7b0e-6803-409d-9c85-f98ed83832e5 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:41:13.304921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-17be7b0e-6803-409d-9c85-f98ed83832e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:41:13.318421 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be7b0e-6803-409d-9c85-f98ed83832e5 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:41:13.324090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:41:13.324751 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:41:13.325028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:41:13.325302 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] PATH_INFO: `/v3/projects/90712f65960b439e92b1e73441412cfa` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:41:13.326505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90712f65960b439e92b1e73441412cfa)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:41:13.328466 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:41:13.508878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90712f65960b439e92b1e73441412cfa', 'request_id': 'req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:41:13.515722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90712f65960b439e92b1e73441412cfa', 'request_id': 'req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:41:13.522984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90712f65960b439e92b1e73441412cfa', 'request_id': 'req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:41:13.661345 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] The token cache is being invalidate because project 90712f65960b439e92b1e73441412cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:41:13.662152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:41:13.662292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:41:13.664415 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1420/2791] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:41:13 2022] DELETE /identity/v3/projects/90712f65960b439e92b1e73441412cfa => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:41:14.929875 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb229ebd-118d-44fa-9a33-14b0ff744c2d service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:41:14.931465 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-bb229ebd-118d-44fa-9a33-14b0ff744c2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:41:14.942910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb229ebd-118d-44fa-9a33-14b0ff744c2d service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:41:14.963856 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:41:14.997367 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc97c1f-1ccf-4f79-b3d9-372b2a0af8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:41:14.999058 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc97c1f-1ccf-4f79-b3d9-372b2a0af8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:41:14.999058 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc97c1f-1ccf-4f79-b3d9-372b2a0af8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:41:14.999058 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc97c1f-1ccf-4f79-b3d9-372b2a0af8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:41:15.000650 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8cc97c1f-1ccf-4f79-b3d9-372b2a0af8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:41:15.073339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc97c1f-1ccf-4f79-b3d9-372b2a0af8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:41:15.074840 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc97c1f-1ccf-4f79-b3d9-372b2a0af8a8 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:41:15.087715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1372/2792] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:41:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:41:23.656526 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:41:25.082556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=40 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:41:38.070197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:41:38.071675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:41:38.138223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e0a8f-b0a0-4c02-afea-0c304dbcbe91 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:41:38.142103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5fa39-97b9-491c-81d9-469a0d4d3f39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:41:38.142764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5fa39-97b9-491c-81d9-469a0d4d3f39 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:41:38.143042 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5fa39-97b9-491c-81d9-469a0d4d3f39 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:41:38.143453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5fa39-97b9-491c-81d9-469a0d4d3f39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:41:38.145289 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bca5fa39-97b9-491c-81d9-469a0d4d3f39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:41:38.226390 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5fa39-97b9-491c-81d9-469a0d4d3f39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:41:38.227787 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5fa39-97b9-491c-81d9-469a0d4d3f39 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:41:38.240365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1421/2793] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:41:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:41:48.234721 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=54 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:41:49.317531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc97c1f-1ccf-4f79-b3d9-372b2a0af8a8 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:41:49.319430 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8cc97c1f-1ccf-4f79-b3d9-372b2a0af8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:41:49.331686 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc97c1f-1ccf-4f79-b3d9-372b2a0af8a8 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:41:49.336765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48613d-d6b5-4374-9dd8-a96ac56311ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:41:49.337390 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48613d-d6b5-4374-9dd8-a96ac56311ff service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:41:49.337663 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48613d-d6b5-4374-9dd8-a96ac56311ff service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:41:49.337929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48613d-d6b5-4374-9dd8-a96ac56311ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:41:49.339820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9a48613d-d6b5-4374-9dd8-a96ac56311ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:41:49.356820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48613d-d6b5-4374-9dd8-a96ac56311ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:41:49.358288 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48613d-d6b5-4374-9dd8-a96ac56311ff service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:41:49.371920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1373/2794] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:41:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:41:59.365392 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=6 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:42:21.350090 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5fa39-97b9-491c-81d9-469a0d4d3f39 service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:42:21.351733 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-bca5fa39-97b9-491c-81d9-469a0d4d3f39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:42:21.364560 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5fa39-97b9-491c-81d9-469a0d4d3f39 service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:42:21.368385 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355b50b-6d1c-4bf8-bb2b-a475cbe39e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:42:21.369005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355b50b-6d1c-4bf8-bb2b-a475cbe39e84 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:42:21.369310 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355b50b-6d1c-4bf8-bb2b-a475cbe39e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:42:21.369580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355b50b-6d1c-4bf8-bb2b-a475cbe39e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:42:21.372186 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f355b50b-6d1c-4bf8-bb2b-a475cbe39e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:42:21.442826 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355b50b-6d1c-4bf8-bb2b-a475cbe39e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:42:21.444311 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355b50b-6d1c-4bf8-bb2b-a475cbe39e84 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:42:21.457970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1422/2795] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:42:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:42:22.826305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48613d-d6b5-4374-9dd8-a96ac56311ff service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:42:22.836538 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48613d-d6b5-4374-9dd8-a96ac56311ff service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:42:22.840009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558f3d7-b38d-4603-ba5e-8f7c4dd363ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:42:22.840623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558f3d7-b38d-4603-ba5e-8f7c4dd363ec service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:42:22.840898 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558f3d7-b38d-4603-ba5e-8f7c4dd363ec service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:42:22.841163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558f3d7-b38d-4603-ba5e-8f7c4dd363ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:42:22.843049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7558f3d7-b38d-4603-ba5e-8f7c4dd363ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:42:22.854417 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558f3d7-b38d-4603-ba5e-8f7c4dd363ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:42:22.856844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558f3d7-b38d-4603-ba5e-8f7c4dd363ec service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:42:22.867159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1374/2796] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:42:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:42:28.478368 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355b50b-6d1c-4bf8-bb2b-a475cbe39e84 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:42:28.491359 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355b50b-6d1c-4bf8-bb2b-a475cbe39e84 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:42:28.497178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252b0c6-00cb-4b93-98ba-9626abd2fb69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:42:28.498194 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252b0c6-00cb-4b93-98ba-9626abd2fb69 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:42:28.498716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252b0c6-00cb-4b93-98ba-9626abd2fb69 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:42:28.499220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252b0c6-00cb-4b93-98ba-9626abd2fb69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:42:28.501792 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5252b0c6-00cb-4b93-98ba-9626abd2fb69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:42:28.577277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252b0c6-00cb-4b93-98ba-9626abd2fb69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:42:28.578696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252b0c6-00cb-4b93-98ba-9626abd2fb69 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:42:28.588986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1423/2797] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:42:32.554393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558f3d7-b38d-4603-ba5e-8f7c4dd363ec service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:42:32.556126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7558f3d7-b38d-4603-ba5e-8f7c4dd363ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:42:32.641933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558f3d7-b38d-4603-ba5e-8f7c4dd363ec service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:42:32.648988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f798fbe-7a4f-4738-a59e-90971b09ba5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:42:32.650042 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f798fbe-7a4f-4738-a59e-90971b09ba5c service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:42:32.650536 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f798fbe-7a4f-4738-a59e-90971b09ba5c service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:42:32.650931 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f798fbe-7a4f-4738-a59e-90971b09ba5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:42:32.653345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5f798fbe-7a4f-4738-a59e-90971b09ba5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:42:32.670662 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f798fbe-7a4f-4738-a59e-90971b09ba5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:42:32.672305 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f798fbe-7a4f-4738-a59e-90971b09ba5c service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:42:32.685265 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1375/2798] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:42:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:42:36.595451 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252b0c6-00cb-4b93-98ba-9626abd2fb69 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:42:36.608150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252b0c6-00cb-4b93-98ba-9626abd2fb69 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:42:36.614369 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f24337-54e4-4091-8cbd-233761402455 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:42:36.624649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f24337-54e4-4091-8cbd-233761402455 service placement] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:42:36.624649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f24337-54e4-4091-8cbd-233761402455 service placement] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:42:36.624649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f24337-54e4-4091-8cbd-233761402455 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:42:36.624649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-82f24337-54e4-4091-8cbd-233761402455 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:42:36.643648 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f24337-54e4-4091-8cbd-233761402455 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:42:36.646101 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f24337-54e4-4091-8cbd-233761402455 service placement] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:42:36.661757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1424/2799] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:42:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:42:36.739150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f798fbe-7a4f-4738-a59e-90971b09ba5c service placement] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:42:36.741162 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-5f798fbe-7a4f-4738-a59e-90971b09ba5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:42:36.757617 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f798fbe-7a4f-4738-a59e-90971b09ba5c service placement] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:42:36.763165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:42:36.763165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:42:36.763555 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:42:36.763782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] PATH_INFO: `/v3/users/9454a7a736164ff09382babab30694d8` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:42:36.765165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9454a7a736164ff09382babab30694d8)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:42:36.767421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:42:36.992497 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9454a7a736164ff09382babab30694d8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:36.993163 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9454a7a736164ff09382babab30694d8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:36.993776 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9454a7a736164ff09382babab30694d8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:37.001343 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9454a7a736164ff09382babab30694d8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:37.014339 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9454a7a736164ff09382babab30694d8'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:37.121522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1376/2800] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:42:36 2022] DELETE /identity/v3/users/9454a7a736164ff09382babab30694d8 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:42:37.426716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f24337-54e4-4091-8cbd-233761402455 service placement] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:42:37.444585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f24337-54e4-4091-8cbd-233761402455 service placement] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:42:37.514224 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:42:37.514899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:42:37.515179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:42:37.515455 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] PATH_INFO: `/v3/projects/608e3c25b97f4a75b2a80903a9d5c656` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:42:37.516585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=608e3c25b97f4a75b2a80903a9d5c656)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:42:37.519124 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:42:37.543087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=100 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:42:37.679151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '608e3c25b97f4a75b2a80903a9d5c656', 'request_id': 'req-14a76ceb-b43d-4fb8-830d-266c651fe4f3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:37.685288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '608e3c25b97f4a75b2a80903a9d5c656', 'request_id': 'req-14a76ceb-b43d-4fb8-830d-266c651fe4f3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:37.696456 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '608e3c25b97f4a75b2a80903a9d5c656', 'request_id': 'req-14a76ceb-b43d-4fb8-830d-266c651fe4f3'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:37.775558 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] The token cache is being invalidate because project 608e3c25b97f4a75b2a80903a9d5c656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:42:37.775899 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:37.776741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:37.778540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1425/2801] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:42:37 2022] DELETE /identity/v3/projects/608e3c25b97f4a75b2a80903a9d5c656 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:42:37.783720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:42:37.787158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:42:37.802036 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d26ed4-bd74-4f55-a71a-11e465e70a6e admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:42:37.853816 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:42:37.866910 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:42:37.867554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:42:37.867827 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:42:37.868133 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] PATH_INFO: `/v3/users/f26628e45f234578a7d582bca5aede59` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:42:37.869421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f26628e45f234578a7d582bca5aede59)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:42:37.871844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:42:38.055624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f26628e45f234578a7d582bca5aede59'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:38.056246 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f26628e45f234578a7d582bca5aede59'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:38.056896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f26628e45f234578a7d582bca5aede59'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:38.064236 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f26628e45f234578a7d582bca5aede59'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:38.076402 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f26628e45f234578a7d582bca5aede59'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:38.171466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1377/2802] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:42:37 2022] DELETE /identity/v3/users/f26628e45f234578a7d582bca5aede59 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:42:38.442336 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:42:38.447313 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:42:38.458295 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76ceb-b43d-4fb8-830d-266c651fe4f3 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:42:38.521768 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:42:38.522660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:42:38.522951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:42:38.523227 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] PATH_INFO: `/v3/projects/4f5b48501d8c4d55a151a57d1d271d9d` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:42:38.524420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f5b48501d8c4d55a151a57d1d271d9d)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:42:38.527866 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:42:38.665618 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f5b48501d8c4d55a151a57d1d271d9d', 'request_id': 'req-76318e18-05e1-4439-910e-d2b94df1c65f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:38.671986 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f5b48501d8c4d55a151a57d1d271d9d', 'request_id': 'req-76318e18-05e1-4439-910e-d2b94df1c65f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:38.678293 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f5b48501d8c4d55a151a57d1d271d9d', 'request_id': 'req-76318e18-05e1-4439-910e-d2b94df1c65f'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:38.762822 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] The token cache is being invalidate because project 4f5b48501d8c4d55a151a57d1d271d9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:42:38.763524 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:38.764431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:42:38.767246 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1426/2803] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:42:38 2022] DELETE /identity/v3/projects/4f5b48501d8c4d55a151a57d1d271d9d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:42:48.167660 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:42:48.757197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:43:21.218881 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:43:21.221694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:43:21.406000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c7734-7c9f-482e-a5fd-afa644749afd admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:43:21.480492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10da1e5-99dd-48d1-8122-314fdf4051aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:43:21.481119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10da1e5-99dd-48d1-8122-314fdf4051aa service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:43:21.481394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10da1e5-99dd-48d1-8122-314fdf4051aa service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:43:21.481672 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10da1e5-99dd-48d1-8122-314fdf4051aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:43:21.483549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b10da1e5-99dd-48d1-8122-314fdf4051aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:43:21.499744 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10da1e5-99dd-48d1-8122-314fdf4051aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:43:21.502028 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10da1e5-99dd-48d1-8122-314fdf4051aa service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:43:21.518696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1378/2804] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:43:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:43:31.509769 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=84 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:43:34.861299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:43:34.862791 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:43:34.956949 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318e18-05e1-4439-910e-d2b94df1c65f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:43:34.960756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3867f0d-6211-4d3f-8e45-e766f865cc45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:43:34.961366 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3867f0d-6211-4d3f-8e45-e766f865cc45 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:43:34.961646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3867f0d-6211-4d3f-8e45-e766f865cc45 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:43:34.961922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3867f0d-6211-4d3f-8e45-e766f865cc45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:43:34.963796 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d3867f0d-6211-4d3f-8e45-e766f865cc45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:43:35.034681 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3867f0d-6211-4d3f-8e45-e766f865cc45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:43:35.036146 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3867f0d-6211-4d3f-8e45-e766f865cc45 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:43:35.046975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1427/2805] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:43:43.271371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10da1e5-99dd-48d1-8122-314fdf4051aa service nova] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:43:43.271371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-b10da1e5-99dd-48d1-8122-314fdf4051aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:43:43.285914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10da1e5-99dd-48d1-8122-314fdf4051aa service nova] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:43:43.291222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:43:43.291879 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:43:43.292159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:43:43.292432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] PATH_INFO: `/v3/users/aa55b9fade0047c68c2f3f6bf4dfe44e` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:43:43.293630 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa55b9fade0047c68c2f3f6bf4dfe44e)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:43:43.295645 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:43:43.429933 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa55b9fade0047c68c2f3f6bf4dfe44e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:43.430642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa55b9fade0047c68c2f3f6bf4dfe44e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:43.431371 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa55b9fade0047c68c2f3f6bf4dfe44e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:43.439351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa55b9fade0047c68c2f3f6bf4dfe44e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:43.451021 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa55b9fade0047c68c2f3f6bf4dfe44e'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:43.507600 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1379/2806] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:43:43 2022] DELETE /identity/v3/users/aa55b9fade0047c68c2f3f6bf4dfe44e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:43:43.765860 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3867f0d-6211-4d3f-8e45-e766f865cc45 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:43:43.804755 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3867f0d-6211-4d3f-8e45-e766f865cc45 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:43:43.861292 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:43:43.861921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:43:43.862198 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:43:43.862469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] PATH_INFO: `/v3/projects/43d4499a9d9f4c00a09c867bb403dd30` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:43:43.863671 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d4499a9d9f4c00a09c867bb403dd30)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:43:43.865585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:43:43.958979 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:43:43.971947 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d4499a9d9f4c00a09c867bb403dd30', 'request_id': 'req-ea180706-038d-4b1f-a475-13946ec5b5f8'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:43.978372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d4499a9d9f4c00a09c867bb403dd30', 'request_id': 'req-ea180706-038d-4b1f-a475-13946ec5b5f8'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:43.983833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d4499a9d9f4c00a09c867bb403dd30', 'request_id': 'req-ea180706-038d-4b1f-a475-13946ec5b5f8'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:44.074019 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] The token cache is being invalidate because project 43d4499a9d9f4c00a09c867bb403dd30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:43:44.074256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:44.074876 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:44.076473 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1428/2807] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:43:43 2022] DELETE /identity/v3/projects/43d4499a9d9f4c00a09c867bb403dd30 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:43:44.081255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:43:44.082894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:43:44.108107 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c247bad-2ff6-4b34-b62a-0178f4849297 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:43:44.176129 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:43:44.176773 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:43:44.177043 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:43:44.177313 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] PATH_INFO: `/v3/users/d153cab8088641b896e6d4e92b57848c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:43:44.178465 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d153cab8088641b896e6d4e92b57848c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:43:44.180567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:43:44.376628 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd153cab8088641b896e6d4e92b57848c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:44.377320 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd153cab8088641b896e6d4e92b57848c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:44.378319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd153cab8088641b896e6d4e92b57848c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:44.385610 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd153cab8088641b896e6d4e92b57848c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:44.394171 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:43:44.401025 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd153cab8088641b896e6d4e92b57848c'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:44.483886 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1380/2808] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:43:44 2022] DELETE /identity/v3/users/d153cab8088641b896e6d4e92b57848c => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:43:44.782188 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:43:44.786931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:43:44.797723 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea180706-038d-4b1f-a475-13946ec5b5f8 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:43:44.856577 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:43:44.857216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:43:44.857497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:43:44.857769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] PATH_INFO: `/v3/projects/51cb74740c3a44159fc3536014a8dfc8` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:43:44.858952 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] RBAC: Authorizing `identity:delete_project(project_id=51cb74740c3a44159fc3536014a8dfc8)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:43:44.860898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:43:44.964580 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51cb74740c3a44159fc3536014a8dfc8', 'request_id': 'req-ba958c5b-4e72-4568-b1bc-8a5f424bceac'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:44.970851 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51cb74740c3a44159fc3536014a8dfc8', 'request_id': 'req-ba958c5b-4e72-4568-b1bc-8a5f424bceac'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:44.977572 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51cb74740c3a44159fc3536014a8dfc8', 'request_id': 'req-ba958c5b-4e72-4568-b1bc-8a5f424bceac'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:45.060500 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] The token cache is being invalidate because project 51cb74740c3a44159fc3536014a8dfc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:43:45.060746 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:45.061386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:43:45.062972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1429/2809] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:43:44 2022] DELETE /identity/v3/projects/51cb74740c3a44159fc3536014a8dfc8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:43:54.620846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=5,keystone:DELETE=2,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:43:55.057191 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=68,keystone:DELETE=12,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:44:37.829836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:44:37.831988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:37.934084 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b58d69-11fb-4349-be82-e4dd8a107cc6 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:44:37.997315 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551b325-c244-4ab4-8155-8b7f9e986415 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:44:37.997915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551b325-c244-4ab4-8155-8b7f9e986415 service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:44:37.998192 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551b325-c244-4ab4-8155-8b7f9e986415 service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:44:37.998465 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551b325-c244-4ab4-8155-8b7f9e986415 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:44:38.000328 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7551b325-c244-4ab4-8155-8b7f9e986415 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:38.013007 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7551b325-c244-4ab4-8155-8b7f9e986415 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:44:38.014677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7551b325-c244-4ab4-8155-8b7f9e986415 service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:44:38.026318 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1381/2810] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:44:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:44:40.255084 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:44:40.256716 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:40.339298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba958c5b-4e72-4568-b1bc-8a5f424bceac admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:44:40.342972 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dab42-038f-46ac-bf64-67ab400e4f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:44:40.343625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dab42-038f-46ac-bf64-67ab400e4f41 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:44:40.343896 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dab42-038f-46ac-bf64-67ab400e4f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:44:40.344164 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dab42-038f-46ac-bf64-67ab400e4f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:44:40.346081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fa2dab42-038f-46ac-bf64-67ab400e4f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:40.406261 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dab42-038f-46ac-bf64-67ab400e4f41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:44:40.407744 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dab42-038f-46ac-bf64-67ab400e4f41 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:44:40.417919 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1430/2811] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:44:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:44:44.008691 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551b325-c244-4ab4-8155-8b7f9e986415 service placement] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:44:44.010231 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7551b325-c244-4ab4-8155-8b7f9e986415 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:44.123250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=100 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:44:44.177822 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551b325-c244-4ab4-8155-8b7f9e986415 service placement] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:44:44.184126 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05e037-709b-4b8b-8a64-5876af4a9178 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:44:44.184787 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05e037-709b-4b8b-8a64-5876af4a9178 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:44:44.185051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05e037-709b-4b8b-8a64-5876af4a9178 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:44:44.185389 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05e037-709b-4b8b-8a64-5876af4a9178 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:44:44.187510 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ed05e037-709b-4b8b-8a64-5876af4a9178 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:44.206580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05e037-709b-4b8b-8a64-5876af4a9178 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:44:44.208061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05e037-709b-4b8b-8a64-5876af4a9178 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:44:44.222150 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1382/2812] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:44:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:44:50.113743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e21116-47b1-4100-aa66-88f3a514232e None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:44:50.114357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e21116-47b1-4100-aa66-88f3a514232e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:44:50.114357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e21116-47b1-4100-aa66-88f3a514232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:44:50.134573 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f6e21116-47b1-4100-aa66-88f3a514232e None None] MFA Rules not processed for user `5042520b122a4c6c94321a50ea57633a`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:44:50.135782 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f6e21116-47b1-4100-aa66-88f3a514232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:50.143107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1431/2813] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:44:50.464059 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05e037-709b-4b8b-8a64-5876af4a9178 service nova] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:44:50.466121 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-ed05e037-709b-4b8b-8a64-5876af4a9178 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:50.548808 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05e037-709b-4b8b-8a64-5876af4a9178 service nova] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:44:50.552671 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b9f1c-8ee0-4aa8-aad8-9eaf88c8ec61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:44:50.553291 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9f1c-8ee0-4aa8-aad8-9eaf88c8ec61 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:44:50.553568 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9f1c-8ee0-4aa8-aad8-9eaf88c8ec61 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:44:50.553839 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9f1c-8ee0-4aa8-aad8-9eaf88c8ec61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:44:50.555758 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-287b9f1c-8ee0-4aa8-aad8-9eaf88c8ec61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:50.567655 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b9f1c-8ee0-4aa8-aad8-9eaf88c8ec61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:44:50.569209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b9f1c-8ee0-4aa8-aad8-9eaf88c8ec61 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:44:50.578077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1383/2814] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:44:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:44:52.053650 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e21116-47b1-4100-aa66-88f3a514232e None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:44:52.055167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f6e21116-47b1-4100-aa66-88f3a514232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:52.071232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e21116-47b1-4100-aa66-88f3a514232e None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:44:52.078040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc168c-8fa7-4560-bcdb-3a39dd010d28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:44:52.078683 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc168c-8fa7-4560-bcdb-3a39dd010d28 service placement] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:44:52.078956 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc168c-8fa7-4560-bcdb-3a39dd010d28 service placement] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:44:52.079223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc168c-8fa7-4560-bcdb-3a39dd010d28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:44:52.080969 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-62fc168c-8fa7-4560-bcdb-3a39dd010d28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:44:52.093304 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc168c-8fa7-4560-bcdb-3a39dd010d28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:44:52.094942 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc168c-8fa7-4560-bcdb-3a39dd010d28 service placement] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:44:52.105853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1432/2815] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:44:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:45:00.574153 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=48 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:45:02.100833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=66 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:45:05.665263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b9f1c-8ee0-4aa8-aad8-9eaf88c8ec61 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:45:05.680209 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b9f1c-8ee0-4aa8-aad8-9eaf88c8ec61 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:45:05.686996 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69237e1-c051-49e3-8e16-860a728be591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:45:05.688338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69237e1-c051-49e3-8e16-860a728be591 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:45:05.688338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69237e1-c051-49e3-8e16-860a728be591 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:45:05.688338 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69237e1-c051-49e3-8e16-860a728be591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:45:05.690123 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e69237e1-c051-49e3-8e16-860a728be591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:45:05.705358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69237e1-c051-49e3-8e16-860a728be591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:45:05.705358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69237e1-c051-49e3-8e16-860a728be591 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:45:05.717140 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1384/2816] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:45:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:45:15.711904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=6 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:46:46.103984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc168c-8fa7-4560-bcdb-3a39dd010d28 service placement] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:46:46.105503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-62fc168c-8fa7-4560-bcdb-3a39dd010d28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:46:46.117922 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc168c-8fa7-4560-bcdb-3a39dd010d28 service placement] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:46:46.121505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6a7a8-6f58-4e62-95a5-1427d00162c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:46:46.122094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6a7a8-6f58-4e62-95a5-1427d00162c7 service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:46:46.122372 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6a7a8-6f58-4e62-95a5-1427d00162c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:46:46.122672 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6a7a8-6f58-4e62-95a5-1427d00162c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:46:46.124419 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7ac6a7a8-6f58-4e62-95a5-1427d00162c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:46:46.138988 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6a7a8-6f58-4e62-95a5-1427d00162c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:46:46.140454 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6a7a8-6f58-4e62-95a5-1427d00162c7 service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:46:46.152508 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1433/2817] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:46:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:46:52.180831 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69237e1-c051-49e3-8e16-860a728be591 service nova] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:46:52.182424 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e69237e1-c051-49e3-8e16-860a728be591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:46:52.194066 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69237e1-c051-49e3-8e16-860a728be591 service nova] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:46:52.197694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85315cac-5cc7-4def-b3b8-55537c686bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:46:52.198285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85315cac-5cc7-4def-b3b8-55537c686bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:46:52.198592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85315cac-5cc7-4def-b3b8-55537c686bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:46:52.199195 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85315cac-5cc7-4def-b3b8-55537c686bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:46:52.200531 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-85315cac-5cc7-4def-b3b8-55537c686bc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:46:52.214377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85315cac-5cc7-4def-b3b8-55537c686bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:46:52.215884 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85315cac-5cc7-4def-b3b8-55537c686bc3 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:46:52.228520 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1385/2818] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:46:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:46:56.147347 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=6 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:00.822407 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6a7a8-6f58-4e62-95a5-1427d00162c7 service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:00.834401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6a7a8-6f58-4e62-95a5-1427d00162c7 service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:00.838082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcd479-6151-446d-979c-d4d35d55e71b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:00.838732 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcd479-6151-446d-979c-d4d35d55e71b service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:00.839013 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcd479-6151-446d-979c-d4d35d55e71b service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:00.839285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcd479-6151-446d-979c-d4d35d55e71b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:00.841107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-05fcd479-6151-446d-979c-d4d35d55e71b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:00.907786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcd479-6151-446d-979c-d4d35d55e71b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:00.909808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcd479-6151-446d-979c-d4d35d55e71b service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:00.920305 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1434/2819] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:02.222534 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=6 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:04.520346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85315cac-5cc7-4def-b3b8-55537c686bc3 service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:04.523495 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-85315cac-5cc7-4def-b3b8-55537c686bc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:04.544513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85315cac-5cc7-4def-b3b8-55537c686bc3 service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:04.549426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:04.550578 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:04.550858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:04.551128 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] PATH_INFO: `/v3/users/5042520b122a4c6c94321a50ea57633a` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:04.552282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5042520b122a4c6c94321a50ea57633a)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:04.554223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:04.739303 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5042520b122a4c6c94321a50ea57633a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:04.739979 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5042520b122a4c6c94321a50ea57633a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:04.740642 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5042520b122a4c6c94321a50ea57633a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:04.747811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5042520b122a4c6c94321a50ea57633a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:04.767939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-9322d2b1-31c0-484e-8a4a-5152751300eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5042520b122a4c6c94321a50ea57633a'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:04.859991 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1386/2820] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:47:04 2022] DELETE /identity/v3/users/5042520b122a4c6c94321a50ea57633a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:05.124297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcd479-6151-446d-979c-d4d35d55e71b service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:05.136082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcd479-6151-446d-979c-d4d35d55e71b service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:05.205853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:05.207675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:05.207955 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:05.208218 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] PATH_INFO: `/v3/projects/5f7d9ac076ce40b09043ced00f54f33a` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:05.209365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7d9ac076ce40b09043ced00f54f33a)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:05.211297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:05.303103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7d9ac076ce40b09043ced00f54f33a', 'request_id': 'req-19d65ce4-bf98-478d-8034-d718c49addd7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:05.309869 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7d9ac076ce40b09043ced00f54f33a', 'request_id': 'req-19d65ce4-bf98-478d-8034-d718c49addd7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:05.316667 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7d9ac076ce40b09043ced00f54f33a', 'request_id': 'req-19d65ce4-bf98-478d-8034-d718c49addd7'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:05.379001 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] The token cache is being invalidate because project 5f7d9ac076ce40b09043ced00f54f33a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:47:05.379274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:05.380092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-19d65ce4-bf98-478d-8034-d718c49addd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:05.382236 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1435/2821] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:47:05 2022] DELETE /identity/v3/projects/5f7d9ac076ce40b09043ced00f54f33a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:05.387358 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cdac0-f599-45dd-905f-5727ab61242f None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:05.387635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cdac0-f599-45dd-905f-5727ab61242f None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:05.387903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cdac0-f599-45dd-905f-5727ab61242f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:05.448926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-692cdac0-f599-45dd-905f-5727ab61242f None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:05.450176 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-692cdac0-f599-45dd-905f-5727ab61242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:05.522916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1387/2822] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:05.531203 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616790f-2760-46d3-9677-e74be044e17d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:05.531488 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616790f-2760-46d3-9677-e74be044e17d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:05.531833 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616790f-2760-46d3-9677-e74be044e17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:05.552128 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6616790f-2760-46d3-9677-e74be044e17d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:05.553352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6616790f-2760-46d3-9677-e74be044e17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:05.561440 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1436/2823] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:05.566547 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cdac0-f599-45dd-905f-5727ab61242f None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:05.576456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cdac0-f599-45dd-905f-5727ab61242f None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:05.582057 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435d734-a2d9-424b-88fd-3623cb5fb635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:05.582723 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435d734-a2d9-424b-88fd-3623cb5fb635 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:05.582999 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435d734-a2d9-424b-88fd-3623cb5fb635 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:05.587651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435d734-a2d9-424b-88fd-3623cb5fb635 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:05.587651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435d734-a2d9-424b-88fd-3623cb5fb635 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:47:05.587651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435d734-a2d9-424b-88fd-3623cb5fb635 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:05.587651 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435d734-a2d9-424b-88fd-3623cb5fb635 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:05.597950 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1388/2824] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:47:05.606807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616790f-2760-46d3-9677-e74be044e17d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:05.616676 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616790f-2760-46d3-9677-e74be044e17d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:05.621958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31cbbf-07a1-4937-b20a-7418477b8e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:05.622718 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31cbbf-07a1-4937-b20a-7418477b8e72 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:05.623056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31cbbf-07a1-4937-b20a-7418477b8e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:05.623335 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31cbbf-07a1-4937-b20a-7418477b8e72 admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:05.624006 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31cbbf-07a1-4937-b20a-7418477b8e72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:05.625921 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31cbbf-07a1-4937-b20a-7418477b8e72 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:05.642405 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:05.761847 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1437/2825] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:47:05 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:47:05.766275 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435d734-a2d9-424b-88fd-3623cb5fb635 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:05.777477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435d734-a2d9-424b-88fd-3623cb5fb635 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:05.807117 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:05.839844 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ea0e4-bced-4361-805f-2cc0313b28ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:05.840456 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ea0e4-bced-4361-805f-2cc0313b28ee admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:05.840727 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ea0e4-bced-4361-805f-2cc0313b28ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:05.841000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ea0e4-bced-4361-805f-2cc0313b28ee admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:05.841643 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ea0e4-bced-4361-805f-2cc0313b28ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:05.843693 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ea0e4-bced-4361-805f-2cc0313b28ee admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:05.913190 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1389/2826] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:47:05 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:47:05.917494 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31cbbf-07a1-4937-b20a-7418477b8e72 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:05.929657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31cbbf-07a1-4937-b20a-7418477b8e72 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:05.934825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984cca8-a1c3-439d-a5f7-c7da0a7f72db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:05.935453 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984cca8-a1c3-439d-a5f7-c7da0a7f72db admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:05.935748 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984cca8-a1c3-439d-a5f7-c7da0a7f72db admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:05.936003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984cca8-a1c3-439d-a5f7-c7da0a7f72db admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:05.936641 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984cca8-a1c3-439d-a5f7-c7da0a7f72db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:05.938264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984cca8-a1c3-439d-a5f7-c7da0a7f72db admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:05.949980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1438/2827] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:05.954105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ea0e4-bced-4361-805f-2cc0313b28ee admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:05.965173 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ea0e4-bced-4361-805f-2cc0313b28ee admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:05.970115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c454d-df63-4b4d-8b21-f05a99c27a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:05.970757 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c454d-df63-4b4d-8b21-f05a99c27a58 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:05.971027 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c454d-df63-4b4d-8b21-f05a99c27a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:05.971298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c454d-df63-4b4d-8b21-f05a99c27a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:05.972139 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c454d-df63-4b4d-8b21-f05a99c27a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:05.973986 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c454d-df63-4b4d-8b21-f05a99c27a58 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:05.984659 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1390/2828] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:05.995620 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984cca8-a1c3-439d-a5f7-c7da0a7f72db admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:06.008862 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984cca8-a1c3-439d-a5f7-c7da0a7f72db admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:06.014096 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412f709-5da9-47b4-ab02-ba1427ef4422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:06.014813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412f709-5da9-47b4-ab02-ba1427ef4422 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:06.015098 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412f709-5da9-47b4-ab02-ba1427ef4422 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:06.015377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412f709-5da9-47b4-ab02-ba1427ef4422 admin admin] PATH_INFO: `/v3/projects/4044ffec95424b1bbe743387e9fc9ca4/users/07196310fa194526a9d777eb62c502b5/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:06.031730 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412f709-5da9-47b4-ab02-ba1427ef4422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4044ffec95424b1bbe743387e9fc9ca4, user_id=07196310fa194526a9d777eb62c502b5, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:06.034444 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412f709-5da9-47b4-ab02-ba1427ef4422 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:06.113423 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1439/2829] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:47:05 2022] PUT /identity/v3/projects/4044ffec95424b1bbe743387e9fc9ca4/users/07196310fa194526a9d777eb62c502b5/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:06.125335 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0818eee-d22c-45ce-a23b-23c7e3356329 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:06.129076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0818eee-d22c-45ce-a23b-23c7e3356329 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:06.129076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0818eee-d22c-45ce-a23b-23c7e3356329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:06.157915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-a0818eee-d22c-45ce-a23b-23c7e3356329 None None] MFA Rules not processed for user `07196310fa194526a9d777eb62c502b5`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:06.159207 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a0818eee-d22c-45ce-a23b-23c7e3356329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:06.237223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1391/2830] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:06.250704 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412f709-5da9-47b4-ab02-ba1427ef4422 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:06.252781 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-7412f709-5da9-47b4-ab02-ba1427ef4422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:06.333439 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412f709-5da9-47b4-ab02-ba1427ef4422 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:06.339238 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb29263-7b4c-4abb-9790-6e67fcfe8a03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:06.339857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb29263-7b4c-4abb-9790-6e67fcfe8a03 service nova] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:06.340163 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb29263-7b4c-4abb-9790-6e67fcfe8a03 service nova] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:06.340403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb29263-7b4c-4abb-9790-6e67fcfe8a03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:06.349483 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb29263-7b4c-4abb-9790-6e67fcfe8a03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:06.351157 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb29263-7b4c-4abb-9790-6e67fcfe8a03 service nova] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:06.360653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1440/2831] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:16.234249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:16.356240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:16.513208 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0818eee-d22c-45ce-a23b-23c7e3356329 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:16.576345 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0818eee-d22c-45ce-a23b-23c7e3356329 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:16.582087 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:16.582745 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:16.583022 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:16.583290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] PATH_INFO: `/v3/users/07196310fa194526a9d777eb62c502b5` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:16.584444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07196310fa194526a9d777eb62c502b5)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:16.586362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:16.762858 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07196310fa194526a9d777eb62c502b5'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:16.763655 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07196310fa194526a9d777eb62c502b5'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:16.764314 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07196310fa194526a9d777eb62c502b5'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:16.772812 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07196310fa194526a9d777eb62c502b5'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:16.785255 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07196310fa194526a9d777eb62c502b5'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:16.844688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1392/2832] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:47:16 2022] DELETE /identity/v3/users/07196310fa194526a9d777eb62c502b5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:16.849689 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42eb5d-d87e-4456-afcd-dcd6c834a996 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:16.849970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42eb5d-d87e-4456-afcd-dcd6c834a996 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:16.850240 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42eb5d-d87e-4456-afcd-dcd6c834a996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:16.871251 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-db42eb5d-d87e-4456-afcd-dcd6c834a996 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:16.872489 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-db42eb5d-d87e-4456-afcd-dcd6c834a996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:16.935078 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1441/2833] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:16.945369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:16.946854 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:17.007524 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595f39a-79a2-445a-9276-122bf5424801 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:17.011505 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4988344-bd36-4109-92ef-f0facca5d063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:17.012115 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4988344-bd36-4109-92ef-f0facca5d063 service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:17.012396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4988344-bd36-4109-92ef-f0facca5d063 service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:17.012665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4988344-bd36-4109-92ef-f0facca5d063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:17.022987 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4988344-bd36-4109-92ef-f0facca5d063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:17.024432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4988344-bd36-4109-92ef-f0facca5d063 service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:17.034297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1393/2834] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:17.747314 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42eb5d-d87e-4456-afcd-dcd6c834a996 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:17.748872 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-db42eb5d-d87e-4456-afcd-dcd6c834a996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:17.761401 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42eb5d-d87e-4456-afcd-dcd6c834a996 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:17.766317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:17.767052 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:17.767330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:17.767600 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] PATH_INFO: `/v3/projects/4044ffec95424b1bbe743387e9fc9ca4` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:17.768889 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4044ffec95424b1bbe743387e9fc9ca4)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:17.771109 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:17.889844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4044ffec95424b1bbe743387e9fc9ca4', 'request_id': 'req-4bd9a287-8262-483c-8594-98a507be66bc'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:17.897148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4044ffec95424b1bbe743387e9fc9ca4', 'request_id': 'req-4bd9a287-8262-483c-8594-98a507be66bc'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:17.903154 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4044ffec95424b1bbe743387e9fc9ca4', 'request_id': 'req-4bd9a287-8262-483c-8594-98a507be66bc'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:17.985948 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] The token cache is being invalidate because project 4044ffec95424b1bbe743387e9fc9ca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:47:17.986234 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:17.986923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4bd9a287-8262-483c-8594-98a507be66bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:17.988849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1442/2835] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:47:17 2022] DELETE /identity/v3/projects/4044ffec95424b1bbe743387e9fc9ca4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:17.993842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731153a-e3ef-4e7a-ab61-b8724fde0d50 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:17.994564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731153a-e3ef-4e7a-ab61-b8724fde0d50 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:17.994564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731153a-e3ef-4e7a-ab61-b8724fde0d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:18.015037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3731153a-e3ef-4e7a-ab61-b8724fde0d50 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:18.016278 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3731153a-e3ef-4e7a-ab61-b8724fde0d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:18.036151 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:18.085269 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1394/2836] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:18.094095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f7846-79ac-42f6-a066-05c540a2d538 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:18.094380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f7846-79ac-42f6-a066-05c540a2d538 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:18.094687 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f7846-79ac-42f6-a066-05c540a2d538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:18.115377 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-da1f7846-79ac-42f6-a066-05c540a2d538 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:18.116583 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-da1f7846-79ac-42f6-a066-05c540a2d538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:18.124564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1443/2837] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:18.129076 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731153a-e3ef-4e7a-ab61-b8724fde0d50 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:18.138976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731153a-e3ef-4e7a-ab61-b8724fde0d50 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:18.144763 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbc363-dc73-4cc1-85d6-6af230621af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:18.145622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc363-dc73-4cc1-85d6-6af230621af9 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:18.146103 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc363-dc73-4cc1-85d6-6af230621af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:18.146774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc363-dc73-4cc1-85d6-6af230621af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:18.147522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbc363-dc73-4cc1-85d6-6af230621af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:47:18.147861 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbc363-dc73-4cc1-85d6-6af230621af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:18.149514 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbc363-dc73-4cc1-85d6-6af230621af9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:18.162224 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1395/2838] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:47:18.166663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f7846-79ac-42f6-a066-05c540a2d538 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:18.176351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f7846-79ac-42f6-a066-05c540a2d538 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:18.181043 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add95c9-919e-4c12-ac67-925faee5bf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:18.181675 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add95c9-919e-4c12-ac67-925faee5bf0d admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:18.181954 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add95c9-919e-4c12-ac67-925faee5bf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:18.182228 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add95c9-919e-4c12-ac67-925faee5bf0d admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:18.182888 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add95c9-919e-4c12-ac67-925faee5bf0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:18.184836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add95c9-919e-4c12-ac67-925faee5bf0d admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:18.293357 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1444/2839] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:47:18 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:47:18.300550 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbc363-dc73-4cc1-85d6-6af230621af9 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:18.315922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbc363-dc73-4cc1-85d6-6af230621af9 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:18.381566 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145266b-4b30-46bd-8c2a-c38287164f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:18.382174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145266b-4b30-46bd-8c2a-c38287164f7d admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:18.382449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145266b-4b30-46bd-8c2a-c38287164f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:18.382751 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145266b-4b30-46bd-8c2a-c38287164f7d admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:18.383383 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145266b-4b30-46bd-8c2a-c38287164f7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:18.385312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145266b-4b30-46bd-8c2a-c38287164f7d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:18.512548 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1396/2840] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:47:18 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:47:18.516482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add95c9-919e-4c12-ac67-925faee5bf0d admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:18.527586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add95c9-919e-4c12-ac67-925faee5bf0d admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:18.532277 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da4ed8-eb5b-4750-a8a2-6e183b36302f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:18.534552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da4ed8-eb5b-4750-a8a2-6e183b36302f admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:18.534552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da4ed8-eb5b-4750-a8a2-6e183b36302f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:18.534552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da4ed8-eb5b-4750-a8a2-6e183b36302f admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:18.534552 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da4ed8-eb5b-4750-a8a2-6e183b36302f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:18.535741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da4ed8-eb5b-4750-a8a2-6e183b36302f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:18.546123 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1445/2841] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:18.551301 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145266b-4b30-46bd-8c2a-c38287164f7d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:18.562384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145266b-4b30-46bd-8c2a-c38287164f7d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:18.568554 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9efe-7f4b-470c-9a35-9118fcc04308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:18.569164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9efe-7f4b-470c-9a35-9118fcc04308 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:18.569443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9efe-7f4b-470c-9a35-9118fcc04308 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:18.569715 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9efe-7f4b-470c-9a35-9118fcc04308 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:18.570365 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9efe-7f4b-470c-9a35-9118fcc04308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:18.572015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9efe-7f4b-470c-9a35-9118fcc04308 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:18.584498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1397/2842] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:18.588874 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da4ed8-eb5b-4750-a8a2-6e183b36302f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:18.599153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da4ed8-eb5b-4750-a8a2-6e183b36302f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:18.604102 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8e87-a9a5-49f9-96bd-9dbdb1c0a96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:18.604756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8e87-a9a5-49f9-96bd-9dbdb1c0a96e admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:18.605040 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8e87-a9a5-49f9-96bd-9dbdb1c0a96e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:18.605317 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8e87-a9a5-49f9-96bd-9dbdb1c0a96e admin admin] PATH_INFO: `/v3/projects/cb242f6a18f5482bad3d6b72a659c935/users/6e1ea734b6e64656ade6990309e5097c/roles/13accf89c12040508974db47c6905b31` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:18.621081 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe8e87-a9a5-49f9-96bd-9dbdb1c0a96e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb242f6a18f5482bad3d6b72a659c935, user_id=6e1ea734b6e64656ade6990309e5097c, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:18.623750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe8e87-a9a5-49f9-96bd-9dbdb1c0a96e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:18.718827 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1446/2843] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:47:18 2022] PUT /identity/v3/projects/cb242f6a18f5482bad3d6b72a659c935/users/6e1ea734b6e64656ade6990309e5097c/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:18.729739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491745d-b05c-400a-b6f2-340f16388d32 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:18.730014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491745d-b05c-400a-b6f2-340f16388d32 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:18.730283 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491745d-b05c-400a-b6f2-340f16388d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:18.759649 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-9491745d-b05c-400a-b6f2-340f16388d32 None None] MFA Rules not processed for user `6e1ea734b6e64656ade6990309e5097c`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:18.760876 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9491745d-b05c-400a-b6f2-340f16388d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:18.826319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1398/2844] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:18.831400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8e87-a9a5-49f9-96bd-9dbdb1c0a96e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:18.853424 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:18.900813 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8e87-a9a5-49f9-96bd-9dbdb1c0a96e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:18.905474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c210de-710f-4c1b-9677-f04bc2b3fa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:18.906088 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c210de-710f-4c1b-9677-f04bc2b3fa3b admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:18.906362 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c210de-710f-4c1b-9677-f04bc2b3fa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:18.906663 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c210de-710f-4c1b-9677-f04bc2b3fa3b admin admin] PATH_INFO: `/v3/projects` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:18.907291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c210de-710f-4c1b-9677-f04bc2b3fa3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:18.910604 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c210de-710f-4c1b-9677-f04bc2b3fa3b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:18.999543 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1447/2845] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:47:18 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:47:19.004347 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491745d-b05c-400a-b6f2-340f16388d32 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:19.015597 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491745d-b05c-400a-b6f2-340f16388d32 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:19.028767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:19.073376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ecaee-9958-45dc-bb6f-c943bee1eb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:19.074049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ecaee-9958-45dc-bb6f-c943bee1eb6e admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:19.074319 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ecaee-9958-45dc-bb6f-c943bee1eb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:19.074624 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ecaee-9958-45dc-bb6f-c943bee1eb6e admin admin] PATH_INFO: `/v3/users` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:19.075267 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ecaee-9958-45dc-bb6f-c943bee1eb6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:19.077272 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ecaee-9958-45dc-bb6f-c943bee1eb6e admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:19.176346 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1399/2846] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:47:19 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:47:19.180594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c210de-710f-4c1b-9677-f04bc2b3fa3b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:19.192517 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c210de-710f-4c1b-9677-f04bc2b3fa3b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:19.197414 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa8cd6-4cb8-4f35-bc8d-d08ef8fcd41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:19.198088 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa8cd6-4cb8-4f35-bc8d-d08ef8fcd41b admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:19.198348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa8cd6-4cb8-4f35-bc8d-d08ef8fcd41b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:19.198695 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa8cd6-4cb8-4f35-bc8d-d08ef8fcd41b admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:19.199352 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa8cd6-4cb8-4f35-bc8d-d08ef8fcd41b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:19.200984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa8cd6-4cb8-4f35-bc8d-d08ef8fcd41b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:19.214330 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1448/2847] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:19.219264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ecaee-9958-45dc-bb6f-c943bee1eb6e admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:19.230302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ecaee-9958-45dc-bb6f-c943bee1eb6e admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:19.235657 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9742dd-faf1-4b12-a688-7c8876dd7d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:19.236356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9742dd-faf1-4b12-a688-7c8876dd7d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:19.236635 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9742dd-faf1-4b12-a688-7c8876dd7d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:19.237101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9742dd-faf1-4b12-a688-7c8876dd7d01 admin admin] PATH_INFO: `/v3/projects/0cf60934435d40ef9fe9db6393a3076f/users/5a1a2f41f04d48f29d6239f8b61c933d/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:19.255263 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9742dd-faf1-4b12-a688-7c8876dd7d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf60934435d40ef9fe9db6393a3076f, user_id=5a1a2f41f04d48f29d6239f8b61c933d, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:19.257667 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9742dd-faf1-4b12-a688-7c8876dd7d01 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:19.353098 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1400/2848] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:47:19 2022] PUT /identity/v3/projects/0cf60934435d40ef9fe9db6393a3076f/users/5a1a2f41f04d48f29d6239f8b61c933d/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:19.357153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa8cd6-4cb8-4f35-bc8d-d08ef8fcd41b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:19.367605 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa8cd6-4cb8-4f35-bc8d-d08ef8fcd41b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:19.430644 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22229f0c-0d60-4085-9d26-e57eabb22950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:19.431225 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22229f0c-0d60-4085-9d26-e57eabb22950 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:19.431506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22229f0c-0d60-4085-9d26-e57eabb22950 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:19.431763 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22229f0c-0d60-4085-9d26-e57eabb22950 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:19.432448 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22229f0c-0d60-4085-9d26-e57eabb22950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:19.434064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22229f0c-0d60-4085-9d26-e57eabb22950 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:19.445174 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1449/2849] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:19.450032 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9742dd-faf1-4b12-a688-7c8876dd7d01 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:19.461447 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9742dd-faf1-4b12-a688-7c8876dd7d01 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:19.466205 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99b80b-ecaf-4064-b6d3-951f006b37a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:19.466911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99b80b-ecaf-4064-b6d3-951f006b37a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:19.467201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99b80b-ecaf-4064-b6d3-951f006b37a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:19.467470 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99b80b-ecaf-4064-b6d3-951f006b37a0 admin admin] PATH_INFO: `/v3/projects/0cf60934435d40ef9fe9db6393a3076f/users/5a1a2f41f04d48f29d6239f8b61c933d/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:19.469354 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99b80b-ecaf-4064-b6d3-951f006b37a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf60934435d40ef9fe9db6393a3076f, user_id=5a1a2f41f04d48f29d6239f8b61c933d, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:19.472622 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99b80b-ecaf-4064-b6d3-951f006b37a0 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:19.484330 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1401/2850] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:47:19 2022] PUT /identity/v3/projects/0cf60934435d40ef9fe9db6393a3076f/users/5a1a2f41f04d48f29d6239f8b61c933d/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:19.496264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10836881-bdc4-47c5-910a-52df7473a5ee None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:19.503285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10836881-bdc4-47c5-910a-52df7473a5ee None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:19.503285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10836881-bdc4-47c5-910a-52df7473a5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:19.526287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-10836881-bdc4-47c5-910a-52df7473a5ee None None] MFA Rules not processed for user `5a1a2f41f04d48f29d6239f8b61c933d`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:19.527548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-10836881-bdc4-47c5-910a-52df7473a5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:19.608193 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1450/2851] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:19.624178 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99b80b-ecaf-4064-b6d3-951f006b37a0 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:19.626081 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0d99b80b-ecaf-4064-b6d3-951f006b37a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:19.684589 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99b80b-ecaf-4064-b6d3-951f006b37a0 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:19.688490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88083d-6981-4d2b-9232-daf06a46b009 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:19.689091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88083d-6981-4d2b-9232-daf06a46b009 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:19.689369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88083d-6981-4d2b-9232-daf06a46b009 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:19.689636 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88083d-6981-4d2b-9232-daf06a46b009 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:19.700110 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88083d-6981-4d2b-9232-daf06a46b009 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:19.701561 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88083d-6981-4d2b-9232-daf06a46b009 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:19.712378 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1402/2852] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:19.743585 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10836881-bdc4-47c5-910a-52df7473a5ee None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:19.762808 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:19.814714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10836881-bdc4-47c5-910a-52df7473a5ee None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:19.820217 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:19.820878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:19.821155 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:19.821429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] PATH_INFO: `/v3/users/6e1ea734b6e64656ade6990309e5097c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:19.822647 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e1ea734b6e64656ade6990309e5097c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:19.824622 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:20.004420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e1ea734b6e64656ade6990309e5097c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:20.005257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e1ea734b6e64656ade6990309e5097c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:20.005887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e1ea734b6e64656ade6990309e5097c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:20.099740 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e1ea734b6e64656ade6990309e5097c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:20.107559 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e1ea734b6e64656ade6990309e5097c'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:20.126902 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1451/2853] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:47:19 2022] DELETE /identity/v3/users/6e1ea734b6e64656ade6990309e5097c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:20.132211 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94645c58-261b-43a8-b374-706ebfa69c70 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:20.132712 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94645c58-261b-43a8-b374-706ebfa69c70 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:20.132834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94645c58-261b-43a8-b374-706ebfa69c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:20.152441 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-94645c58-261b-43a8-b374-706ebfa69c70 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:20.156161 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-94645c58-261b-43a8-b374-706ebfa69c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:20.220223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1403/2854] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:20.230774 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:20.232266 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:20.299974 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8adc0-1c75-4d4a-b1e0-33ef99e58fcf admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:20.303736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc7c44-b1dc-4834-b03d-a48ad227f4b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:20.304400 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc7c44-b1dc-4834-b03d-a48ad227f4b7 service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:20.304677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc7c44-b1dc-4834-b03d-a48ad227f4b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:20.304951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc7c44-b1dc-4834-b03d-a48ad227f4b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:20.315812 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc7c44-b1dc-4834-b03d-a48ad227f4b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:20.317321 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc7c44-b1dc-4834-b03d-a48ad227f4b7 service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:20.328086 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1452/2855] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:20.742732 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94645c58-261b-43a8-b374-706ebfa69c70 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:20.746353 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-94645c58-261b-43a8-b374-706ebfa69c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:20.758234 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94645c58-261b-43a8-b374-706ebfa69c70 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:20.763114 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:20.763759 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:20.764030 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:20.764323 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] PATH_INFO: `/v3/projects/cb242f6a18f5482bad3d6b72a659c935` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:20.766011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb242f6a18f5482bad3d6b72a659c935)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:20.767652 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:20.777564 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:20.892647 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb242f6a18f5482bad3d6b72a659c935', 'request_id': 'req-025faf7e-fb40-4739-b38b-409fbac83a26'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:20.900777 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb242f6a18f5482bad3d6b72a659c935', 'request_id': 'req-025faf7e-fb40-4739-b38b-409fbac83a26'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:20.986625 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb242f6a18f5482bad3d6b72a659c935', 'request_id': 'req-025faf7e-fb40-4739-b38b-409fbac83a26'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:20.998410 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] The token cache is being invalidate because project cb242f6a18f5482bad3d6b72a659c935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:47:20.998723 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:20.999477 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:21.001000 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1404/2856] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:47:20 2022] DELETE /identity/v3/projects/cb242f6a18f5482bad3d6b72a659c935 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:21.005737 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc7c44-b1dc-4834-b03d-a48ad227f4b7 service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:21.016025 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc7c44-b1dc-4834-b03d-a48ad227f4b7 service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:21.051060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:21.080264 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:21.080890 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:21.082438 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:21.082951 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] PATH_INFO: `/v3/users/5a1a2f41f04d48f29d6239f8b61c933d` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:21.084529 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a1a2f41f04d48f29d6239f8b61c933d)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:21.086531 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:21.218673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a1a2f41f04d48f29d6239f8b61c933d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:21.219355 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a1a2f41f04d48f29d6239f8b61c933d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:21.219961 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a1a2f41f04d48f29d6239f8b61c933d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:21.288178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a1a2f41f04d48f29d6239f8b61c933d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:21.296135 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-357b1632-b92a-4bc8-b317-71d4b92f0fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a1a2f41f04d48f29d6239f8b61c933d'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:21.317404 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1453/2857] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:47:21 2022] DELETE /identity/v3/users/5a1a2f41f04d48f29d6239f8b61c933d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:47:21.844778 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:21.846174 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:21.860271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025faf7e-fb40-4739-b38b-409fbac83a26 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:21.922463 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:21.923119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:21.923396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:21.923666 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] PATH_INFO: `/v3/projects/0cf60934435d40ef9fe9db6393a3076f` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:21.924929 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf60934435d40ef9fe9db6393a3076f)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:21.926890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:22.027896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf60934435d40ef9fe9db6393a3076f', 'request_id': 'req-5e5c70c9-9a28-421b-8f44-f408092747f9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:22.034737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf60934435d40ef9fe9db6393a3076f', 'request_id': 'req-5e5c70c9-9a28-421b-8f44-f408092747f9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:22.083453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf60934435d40ef9fe9db6393a3076f', 'request_id': 'req-5e5c70c9-9a28-421b-8f44-f408092747f9'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:22.095250 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] The token cache is being invalidate because project 0cf60934435d40ef9fe9db6393a3076f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:47:22.095454 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:22.096090 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-5e5c70c9-9a28-421b-8f44-f408092747f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:22.097740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1405/2858] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:47:21 2022] DELETE /identity/v3/projects/0cf60934435d40ef9fe9db6393a3076f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:22.103707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b297b-3315-4331-8602-6b390d1b030d None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:22.104260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b297b-3315-4331-8602-6b390d1b030d None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:22.104260 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b297b-3315-4331-8602-6b390d1b030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:22.125680 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-751b297b-3315-4331-8602-6b390d1b030d None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:22.127082 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-751b297b-3315-4331-8602-6b390d1b030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:22.189371 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1454/2859] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 17:47:22.198418 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e569c-fc0e-4d96-a66d-89265203a3bd None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:22.198722 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e569c-fc0e-4d96-a66d-89265203a3bd None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:22.199010 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e569c-fc0e-4d96-a66d-89265203a3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:22.219484 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-7b8e569c-fc0e-4d96-a66d-89265203a3bd None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:22.220706 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-7b8e569c-fc0e-4d96-a66d-89265203a3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:22.229304 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1406/2860] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:22.233696 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b297b-3315-4331-8602-6b390d1b030d None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:22.244859 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b297b-3315-4331-8602-6b390d1b030d None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:22.249649 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dab22-c288-4f32-ae31-b303c3f90112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:22.250296 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dab22-c288-4f32-ae31-b303c3f90112 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:22.250863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dab22-c288-4f32-ae31-b303c3f90112 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:22.251143 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dab22-c288-4f32-ae31-b303c3f90112 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:22.251832 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70dab22-c288-4f32-ae31-b303c3f90112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:47:22.252138 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70dab22-c288-4f32-ae31-b303c3f90112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:22.254653 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70dab22-c288-4f32-ae31-b303c3f90112 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:22.266793 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1455/2861] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:47:22.272049 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e569c-fc0e-4d96-a66d-89265203a3bd None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:22.281626 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e569c-fc0e-4d96-a66d-89265203a3bd None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:22.287930 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e0dbf-ad3a-41c8-86c7-94f310cb003c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:22.288543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0dbf-ad3a-41c8-86c7-94f310cb003c admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:22.288813 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0dbf-ad3a-41c8-86c7-94f310cb003c admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:22.289077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0dbf-ad3a-41c8-86c7-94f310cb003c admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:22.289731 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e0dbf-ad3a-41c8-86c7-94f310cb003c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:22.291688 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e0dbf-ad3a-41c8-86c7-94f310cb003c admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:22.358449 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:22.367870 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1407/2862] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:47:22 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:47:22.372233 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dab22-c288-4f32-ae31-b303c3f90112 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:22.383344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dab22-c288-4f32-ae31-b303c3f90112 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:22.429756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:22.445469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166995b1-f1f5-49a2-8de5-2370944ceae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:22.446089 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166995b1-f1f5-49a2-8de5-2370944ceae9 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:22.446368 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166995b1-f1f5-49a2-8de5-2370944ceae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:22.446697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166995b1-f1f5-49a2-8de5-2370944ceae9 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:22.447396 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166995b1-f1f5-49a2-8de5-2370944ceae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:22.449643 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166995b1-f1f5-49a2-8de5-2370944ceae9 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:22.529063 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1456/2863] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:47:22 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:47:22.533396 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e0dbf-ad3a-41c8-86c7-94f310cb003c admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:22.546914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e0dbf-ad3a-41c8-86c7-94f310cb003c admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:22.553487 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb56581-a9b7-4fb7-85ee-c95587e7104a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:22.553873 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb56581-a9b7-4fb7-85ee-c95587e7104a admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:22.557130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb56581-a9b7-4fb7-85ee-c95587e7104a admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:22.557130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb56581-a9b7-4fb7-85ee-c95587e7104a admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:22.557130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb56581-a9b7-4fb7-85ee-c95587e7104a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:22.557130 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb56581-a9b7-4fb7-85ee-c95587e7104a admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:22.568335 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1408/2864] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:22.572783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166995b1-f1f5-49a2-8de5-2370944ceae9 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:22.584005 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166995b1-f1f5-49a2-8de5-2370944ceae9 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:22.589923 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64686f8-d675-48c7-9e57-037457a4006c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:22.590714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64686f8-d675-48c7-9e57-037457a4006c admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:22.590999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64686f8-d675-48c7-9e57-037457a4006c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:22.591271 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64686f8-d675-48c7-9e57-037457a4006c admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:22.591911 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64686f8-d675-48c7-9e57-037457a4006c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:22.593507 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64686f8-d675-48c7-9e57-037457a4006c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:22.605592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1457/2865] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:22.613466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb56581-a9b7-4fb7-85ee-c95587e7104a admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:22.621325 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb56581-a9b7-4fb7-85ee-c95587e7104a admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:22.626273 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18703275-ea0b-42f3-98c0-bc9c66c21e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:22.626976 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18703275-ea0b-42f3-98c0-bc9c66c21e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:22.627253 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18703275-ea0b-42f3-98c0-bc9c66c21e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:22.627521 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18703275-ea0b-42f3-98c0-bc9c66c21e35 admin admin] PATH_INFO: `/v3/projects/fdf627d221de4ee7804114ed1296494b/users/0db5c00c1aee45ff9731eebb01495911/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:22.642811 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18703275-ea0b-42f3-98c0-bc9c66c21e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf627d221de4ee7804114ed1296494b, user_id=0db5c00c1aee45ff9731eebb01495911, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:22.645674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18703275-ea0b-42f3-98c0-bc9c66c21e35 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:22.718285 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1409/2866] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:47:22 2022] PUT /identity/v3/projects/fdf627d221de4ee7804114ed1296494b/users/0db5c00c1aee45ff9731eebb01495911/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:22.729892 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49093af8-2226-4bcd-a12c-a3bbc3c04dfc None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:22.730208 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49093af8-2226-4bcd-a12c-a3bbc3c04dfc None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:22.730506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49093af8-2226-4bcd-a12c-a3bbc3c04dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:22.760197 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-49093af8-2226-4bcd-a12c-a3bbc3c04dfc None None] MFA Rules not processed for user `0db5c00c1aee45ff9731eebb01495911`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:22.761418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-49093af8-2226-4bcd-a12c-a3bbc3c04dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:22.835540 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1458/2867] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:22.840362 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18703275-ea0b-42f3-98c0-bc9c66c21e35 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:22.912298 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18703275-ea0b-42f3-98c0-bc9c66c21e35 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:22.917709 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ee9a2-6042-4f48-bb0a-6395a724366d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:22.918332 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ee9a2-6042-4f48-bb0a-6395a724366d admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:22.918640 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ee9a2-6042-4f48-bb0a-6395a724366d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:22.918911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ee9a2-6042-4f48-bb0a-6395a724366d admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:22.919553 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ee9a2-6042-4f48-bb0a-6395a724366d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:22.921508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ee9a2-6042-4f48-bb0a-6395a724366d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:23.029939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1410/2868] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:47:22 2022] POST /identity/v3/projects => generated 381 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 17:47:23.034279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49093af8-2226-4bcd-a12c-a3bbc3c04dfc None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:23.047126 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49093af8-2226-4bcd-a12c-a3bbc3c04dfc None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:23.109637 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f08a9-65f7-4354-ba48-4576d396d7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:23.110497 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f08a9-65f7-4354-ba48-4576d396d7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:23.110625 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f08a9-65f7-4354-ba48-4576d396d7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:23.110816 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f08a9-65f7-4354-ba48-4576d396d7e5 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:23.112007 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f08a9-65f7-4354-ba48-4576d396d7e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:23.114714 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f08a9-65f7-4354-ba48-4576d396d7e5 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:23.204503 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1459/2869] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:47:23 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:47:23.209423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ee9a2-6042-4f48-bb0a-6395a724366d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:23.220514 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ee9a2-6042-4f48-bb0a-6395a724366d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:23.226159 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71349c90-68ca-43d6-8abe-e20a5657bfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:23.226809 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71349c90-68ca-43d6-8abe-e20a5657bfcd admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:23.227091 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71349c90-68ca-43d6-8abe-e20a5657bfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:23.227363 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71349c90-68ca-43d6-8abe-e20a5657bfcd admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:23.228014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71349c90-68ca-43d6-8abe-e20a5657bfcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:23.229644 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71349c90-68ca-43d6-8abe-e20a5657bfcd admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:23.242376 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1411/2870] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:23.246635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f08a9-65f7-4354-ba48-4576d396d7e5 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:23.258015 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f08a9-65f7-4354-ba48-4576d396d7e5 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:23.265657 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95450c-7f86-4ddd-b161-d1ff201ff368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:23.266878 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95450c-7f86-4ddd-b161-d1ff201ff368 admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:23.267351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95450c-7f86-4ddd-b161-d1ff201ff368 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:23.267843 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95450c-7f86-4ddd-b161-d1ff201ff368 admin admin] PATH_INFO: `/v3/projects/6eafe4841b9449a28e2cb1751ad3e32c/users/d10eb1909824428ab4990e415c1f2804/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:23.287708 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95450c-7f86-4ddd-b161-d1ff201ff368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eafe4841b9449a28e2cb1751ad3e32c, user_id=d10eb1909824428ab4990e415c1f2804, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:23.290457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95450c-7f86-4ddd-b161-d1ff201ff368 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:23.364504 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1460/2871] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:47:23 2022] PUT /identity/v3/projects/6eafe4841b9449a28e2cb1751ad3e32c/users/d10eb1909824428ab4990e415c1f2804/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:23.368529 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71349c90-68ca-43d6-8abe-e20a5657bfcd admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:23.379750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71349c90-68ca-43d6-8abe-e20a5657bfcd admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:23.439262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24426-d415-49b7-8ffe-1a9c375b2b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:23.439866 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24426-d415-49b7-8ffe-1a9c375b2b08 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:23.440646 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24426-d415-49b7-8ffe-1a9c375b2b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:23.440896 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24426-d415-49b7-8ffe-1a9c375b2b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:23.441944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f24426-d415-49b7-8ffe-1a9c375b2b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:23.443571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f24426-d415-49b7-8ffe-1a9c375b2b08 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:23.454230 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1412/2872] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:23.459291 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95450c-7f86-4ddd-b161-d1ff201ff368 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:23.470223 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95450c-7f86-4ddd-b161-d1ff201ff368 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:23.475150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6cd31-7e68-491d-b7ce-c58ad769c18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:23.475834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6cd31-7e68-491d-b7ce-c58ad769c18b admin admin] REQUEST_METHOD: `PUT` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:23.476122 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6cd31-7e68-491d-b7ce-c58ad769c18b admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:23.476505 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6cd31-7e68-491d-b7ce-c58ad769c18b admin admin] PATH_INFO: `/v3/projects/6eafe4841b9449a28e2cb1751ad3e32c/users/d10eb1909824428ab4990e415c1f2804/roles/4b10782d99b74f5bb453b82286012e0c` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:23.478520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6cd31-7e68-491d-b7ce-c58ad769c18b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eafe4841b9449a28e2cb1751ad3e32c, user_id=d10eb1909824428ab4990e415c1f2804, role_id=4b10782d99b74f5bb453b82286012e0c)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:23.481195 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6cd31-7e68-491d-b7ce-c58ad769c18b admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:23.493857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1461/2873] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:47:23 2022] PUT /identity/v3/projects/6eafe4841b9449a28e2cb1751ad3e32c/users/d10eb1909824428ab4990e415c1f2804/roles/4b10782d99b74f5bb453b82286012e0c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:23.561077 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578c731-64a8-4ad0-977e-a56d9152f968 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:23.561351 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578c731-64a8-4ad0-977e-a56d9152f968 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:23.561629 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578c731-64a8-4ad0-977e-a56d9152f968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:23.591770 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-e578c731-64a8-4ad0-977e-a56d9152f968 None None] MFA Rules not processed for user `d10eb1909824428ab4990e415c1f2804`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:23.593003 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e578c731-64a8-4ad0-977e-a56d9152f968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:23.615393 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:23.680923 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1413/2874] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:23.692949 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6cd31-7e68-491d-b7ce-c58ad769c18b admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:23.694750 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-1dc6cd31-7e68-491d-b7ce-c58ad769c18b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:23.701344 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:23.754958 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6cd31-7e68-491d-b7ce-c58ad769c18b admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:23.759290 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b5837-6a38-417a-bcbc-87bdb34c99de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:23.759898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b5837-6a38-417a-bcbc-87bdb34c99de service cinder] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:23.760167 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b5837-6a38-417a-bcbc-87bdb34c99de service cinder] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:23.760429 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b5837-6a38-417a-bcbc-87bdb34c99de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:23.769877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b5837-6a38-417a-bcbc-87bdb34c99de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:23.771360 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b5837-6a38-417a-bcbc-87bdb34c99de service cinder] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:23.781367 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1462/2875] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:23.888807 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578c731-64a8-4ad0-977e-a56d9152f968 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:23.900113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578c731-64a8-4ad0-977e-a56d9152f968 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:23.903851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5077132-ba59-462d-90cc-a43e055b67b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:23.904462 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5077132-ba59-462d-90cc-a43e055b67b7 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:23.904740 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5077132-ba59-462d-90cc-a43e055b67b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:23.905011 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5077132-ba59-462d-90cc-a43e055b67b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:23.964720 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5077132-ba59-462d-90cc-a43e055b67b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:23.966222 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5077132-ba59-462d-90cc-a43e055b67b7 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:23.978051 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1414/2876] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:47:27.055980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b5837-6a38-417a-bcbc-87bdb34c99de service cinder] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:27.118786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b5837-6a38-417a-bcbc-87bdb34c99de service cinder] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:27.123654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:27.124288 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:27.124569 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:27.124844 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] PATH_INFO: `/v3/users/0db5c00c1aee45ff9731eebb01495911` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:27.125968 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db5c00c1aee45ff9731eebb01495911)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:27.128060 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:27.365520 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db5c00c1aee45ff9731eebb01495911'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:27.366150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db5c00c1aee45ff9731eebb01495911'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:27.366789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db5c00c1aee45ff9731eebb01495911'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:27.448980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db5c00c1aee45ff9731eebb01495911'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:27.457470 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db5c00c1aee45ff9731eebb01495911'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:27.476348 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1463/2877] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:47:27 2022] DELETE /identity/v3/users/0db5c00c1aee45ff9731eebb01495911 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:27.480953 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c868c-4594-4b75-b675-b6737fdeafd5 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:27.481476 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c868c-4594-4b75-b675-b6737fdeafd5 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:27.481476 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c868c-4594-4b75-b675-b6737fdeafd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:27.501532 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-509c868c-4594-4b75-b675-b6737fdeafd5 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:27.502954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-509c868c-4594-4b75-b675-b6737fdeafd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:27.562911 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1415/2878] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:27.573144 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:27.574592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:27.606743 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:27.634380 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367757ed-e67a-469c-9437-0cc1ca4d9a1f admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:27.638041 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a36cf-e00c-4250-8bed-f7e573ee371a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:27.638685 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a36cf-e00c-4250-8bed-f7e573ee371a service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:27.638962 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a36cf-e00c-4250-8bed-f7e573ee371a service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:27.639235 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a36cf-e00c-4250-8bed-f7e573ee371a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:27.649999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a36cf-e00c-4250-8bed-f7e573ee371a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:27.651479 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a36cf-e00c-4250-8bed-f7e573ee371a service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:27.661331 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1464/2879] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:28.090978 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c868c-4594-4b75-b675-b6737fdeafd5 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:28.092432 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-509c868c-4594-4b75-b675-b6737fdeafd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:28.107002 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c868c-4594-4b75-b675-b6737fdeafd5 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:28.112665 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:28.113678 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:28.113867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:28.114075 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] PATH_INFO: `/v3/projects/fdf627d221de4ee7804114ed1296494b` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:28.115287 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf627d221de4ee7804114ed1296494b)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:28.117271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:28.226015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf627d221de4ee7804114ed1296494b', 'request_id': 'req-e848fe52-f033-4512-8143-ec14e096a9d3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:28.228013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:28.232696 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf627d221de4ee7804114ed1296494b', 'request_id': 'req-e848fe52-f033-4512-8143-ec14e096a9d3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:28.318282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf627d221de4ee7804114ed1296494b', 'request_id': 'req-e848fe52-f033-4512-8143-ec14e096a9d3'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:28.331724 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] The token cache is being invalidate because project fdf627d221de4ee7804114ed1296494b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:47:28.332013 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:28.332687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:28.334297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1416/2880] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:47:28 2022] DELETE /identity/v3/projects/fdf627d221de4ee7804114ed1296494b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:28.338633 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a36cf-e00c-4250-8bed-f7e573ee371a service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:28.349362 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a36cf-e00c-4250-8bed-f7e573ee371a service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:28.409472 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:28.410153 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:28.410430 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:28.410727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] PATH_INFO: `/v3/users/d10eb1909824428ab4990e415c1f2804` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:28.412087 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d10eb1909824428ab4990e415c1f2804)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:28.413999 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:28.584110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd10eb1909824428ab4990e415c1f2804'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:28.584700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd10eb1909824428ab4990e415c1f2804'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:28.585313 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd10eb1909824428ab4990e415c1f2804'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:28.661048 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd10eb1909824428ab4990e415c1f2804'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:28.669145 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-4a3aadb0-c3ee-42ea-84be-b228d09c01a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd10eb1909824428ab4990e415c1f2804'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:28.687249 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1465/2881] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:47:28 2022] DELETE /identity/v3/users/d10eb1909824428ab4990e415c1f2804 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:29.120271 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:29.121850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:29.138097 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848fe52-f033-4512-8143-ec14e096a9d3 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:29.202508 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:29.203165 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:29.203442 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:29.203716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] PATH_INFO: `/v3/projects/6eafe4841b9449a28e2cb1751ad3e32c` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:29.204863 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eafe4841b9449a28e2cb1751ad3e32c)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:29.206954 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:29.317152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eafe4841b9449a28e2cb1751ad3e32c', 'request_id': 'req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:29.323455 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eafe4841b9449a28e2cb1751ad3e32c', 'request_id': 'req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:29.403490 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eafe4841b9449a28e2cb1751ad3e32c', 'request_id': 'req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:29.414903 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] The token cache is being invalidate because project 6eafe4841b9449a28e2cb1751ad3e32c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:47:29.415204 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:29.415851 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-fa6a7ac6-1e1d-43c5-8b53-ca5d93ee91e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:29.417384 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1417/2882] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:47:29 2022] DELETE /identity/v3/projects/6eafe4841b9449a28e2cb1751ad3e32c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:29.422356 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5977a07-7b15-4bf7-b317-0f7c0bfb1013 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:29.422914 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5977a07-7b15-4bf7-b317-0f7c0bfb1013 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:29.423114 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5977a07-7b15-4bf7-b317-0f7c0bfb1013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:29.444179 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-f5977a07-7b15-4bf7-b317-0f7c0bfb1013 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:29.445392 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-f5977a07-7b15-4bf7-b317-0f7c0bfb1013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:29.492654 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:29.509700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1466/2883] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:29.517359 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab56918-0ef8-4e96-8619-ca055919d9e2 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:29.523143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab56918-0ef8-4e96-8619-ca055919d9e2 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:29.523143 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab56918-0ef8-4e96-8619-ca055919d9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:29.539266 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3ab56918-0ef8-4e96-8619-ca055919d9e2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:29.546286 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3ab56918-0ef8-4e96-8619-ca055919d9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:29.548400 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1418/2884] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:29.553323 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5977a07-7b15-4bf7-b317-0f7c0bfb1013 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:29.563152 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5977a07-7b15-4bf7-b317-0f7c0bfb1013 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:29.568493 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02eb7e-fc8c-4068-aaf0-efd35c174c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:29.569095 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02eb7e-fc8c-4068-aaf0-efd35c174c13 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:29.569373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02eb7e-fc8c-4068-aaf0-efd35c174c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:29.569645 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02eb7e-fc8c-4068-aaf0-efd35c174c13 admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:29.570287 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02eb7e-fc8c-4068-aaf0-efd35c174c13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:47:29.570629 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02eb7e-fc8c-4068-aaf0-efd35c174c13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:29.572274 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02eb7e-fc8c-4068-aaf0-efd35c174c13 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:29.584557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1467/2885] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:47:29.588975 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab56918-0ef8-4e96-8619-ca055919d9e2 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:29.599506 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab56918-0ef8-4e96-8619-ca055919d9e2 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:29.604697 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d3765-fd3b-450c-9171-057ccf791fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:29.605302 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d3765-fd3b-450c-9171-057ccf791fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:29.605575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d3765-fd3b-450c-9171-057ccf791fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:29.605841 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d3765-fd3b-450c-9171-057ccf791fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:29.606590 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d3765-fd3b-450c-9171-057ccf791fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:29.608499 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d3765-fd3b-450c-9171-057ccf791fb9 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:29.724127 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1419/2886] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:47:29 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:47:29.728512 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02eb7e-fc8c-4068-aaf0-efd35c174c13 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:29.739535 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02eb7e-fc8c-4068-aaf0-efd35c174c13 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:29.797460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b4156-d57b-410b-a48f-d91b02126609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:29.798064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b4156-d57b-410b-a48f-d91b02126609 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:29.798342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b4156-d57b-410b-a48f-d91b02126609 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:29.798646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b4156-d57b-410b-a48f-d91b02126609 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:29.799910 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b4156-d57b-410b-a48f-d91b02126609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:29.801825 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b4156-d57b-410b-a48f-d91b02126609 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:29.935697 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1468/2887] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:47:29 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:47:29.939983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d3765-fd3b-450c-9171-057ccf791fb9 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:29.950623 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d3765-fd3b-450c-9171-057ccf791fb9 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:29.955426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf84fd1-0578-4ea9-8f1b-ca0be182085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:29.956073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf84fd1-0578-4ea9-8f1b-ca0be182085c admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:29.956356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf84fd1-0578-4ea9-8f1b-ca0be182085c admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:29.956957 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf84fd1-0578-4ea9-8f1b-ca0be182085c admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:29.957618 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf84fd1-0578-4ea9-8f1b-ca0be182085c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:29.959289 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf84fd1-0578-4ea9-8f1b-ca0be182085c admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:29.970922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1420/2888] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:29.976028 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b4156-d57b-410b-a48f-d91b02126609 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:29.986511 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b4156-d57b-410b-a48f-d91b02126609 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:29.992427 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9227b5-96b1-4972-b828-ac1c09f3655a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:29.993062 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9227b5-96b1-4972-b828-ac1c09f3655a admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:29.993343 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9227b5-96b1-4972-b828-ac1c09f3655a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:29.993619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9227b5-96b1-4972-b828-ac1c09f3655a admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:29.994280 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9227b5-96b1-4972-b828-ac1c09f3655a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:29.996030 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9227b5-96b1-4972-b828-ac1c09f3655a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:30.008150 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1469/2889] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:30.012567 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf84fd1-0578-4ea9-8f1b-ca0be182085c admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:30.023295 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf84fd1-0578-4ea9-8f1b-ca0be182085c admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:30.027961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6bab5-e14c-44fb-8812-0381406aae9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:30.028605 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6bab5-e14c-44fb-8812-0381406aae9b admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:30.028882 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6bab5-e14c-44fb-8812-0381406aae9b admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:30.029152 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6bab5-e14c-44fb-8812-0381406aae9b admin admin] PATH_INFO: `/v3/projects/5439871b946842c18cd47f5b9f1f9545/users/cf5debc09ea04e019d3136f24eba8196/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:30.044985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6bab5-e14c-44fb-8812-0381406aae9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5439871b946842c18cd47f5b9f1f9545, user_id=cf5debc09ea04e019d3136f24eba8196, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:30.047682 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6bab5-e14c-44fb-8812-0381406aae9b admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:30.192889 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1421/2890] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:47:30 2022] PUT /identity/v3/projects/5439871b946842c18cd47f5b9f1f9545/users/cf5debc09ea04e019d3136f24eba8196/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:30.203398 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925ce9-18d2-4f83-bdd7-e62feb427f96 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:30.203882 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925ce9-18d2-4f83-bdd7-e62feb427f96 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:30.204412 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925ce9-18d2-4f83-bdd7-e62feb427f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:30.246070 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-6e925ce9-18d2-4f83-bdd7-e62feb427f96 None None] MFA Rules not processed for user `cf5debc09ea04e019d3136f24eba8196`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:30.247232 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6e925ce9-18d2-4f83-bdd7-e62feb427f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:30.325798 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1470/2891] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:30.338575 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6bab5-e14c-44fb-8812-0381406aae9b admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:30.340321 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-9ce6bab5-e14c-44fb-8812-0381406aae9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:30.350997 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:30.405960 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6bab5-e14c-44fb-8812-0381406aae9b admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:30.411232 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4807edc-5e43-4e7a-a770-73afe3132723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:30.411846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4807edc-5e43-4e7a-a770-73afe3132723 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:30.412195 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4807edc-5e43-4e7a-a770-73afe3132723 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:30.412694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4807edc-5e43-4e7a-a770-73afe3132723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:30.422453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4807edc-5e43-4e7a-a770-73afe3132723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:30.424342 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4807edc-5e43-4e7a-a770-73afe3132723 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:30.435009 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1422/2892] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:33.610741 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e925ce9-18d2-4f83-bdd7-e62feb427f96 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:33.665924 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:33.685722 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e925ce9-18d2-4f83-bdd7-e62feb427f96 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:33.690772 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:33.691481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:33.691764 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:33.692039 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] PATH_INFO: `/v3/users/cf5debc09ea04e019d3136f24eba8196` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:33.693250 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5debc09ea04e019d3136f24eba8196)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:33.695221 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:33.898332 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5debc09ea04e019d3136f24eba8196'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:33.899050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5debc09ea04e019d3136f24eba8196'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:33.899673 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5debc09ea04e019d3136f24eba8196'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:33.992346 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5debc09ea04e019d3136f24eba8196'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:34.000809 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5debc09ea04e019d3136f24eba8196'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:34.020297 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1471/2893] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:47:33 2022] DELETE /identity/v3/users/cf5debc09ea04e019d3136f24eba8196 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:34.025118 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4d7f9-0ac5-455a-ad6e-f508bc68bbd7 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:34.025595 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4d7f9-0ac5-455a-ad6e-f508bc68bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:34.025716 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4d7f9-0ac5-455a-ad6e-f508bc68bbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:34.044369 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-45d4d7f9-0ac5-455a-ad6e-f508bc68bbd7 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:34.045581 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-45d4d7f9-0ac5-455a-ad6e-f508bc68bbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:34.105631 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1423/2894] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:34.116980 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:34.118726 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:34.180798 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569368b4-1439-489e-bb1a-b8e8f2a1750a admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:34.184485 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ba391-e1f6-47fd-b510-21a50d14490e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:34.185108 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ba391-e1f6-47fd-b510-21a50d14490e service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:34.185386 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ba391-e1f6-47fd-b510-21a50d14490e service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:34.185660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ba391-e1f6-47fd-b510-21a50d14490e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:34.196299 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ba391-e1f6-47fd-b510-21a50d14490e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:34.197769 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ba391-e1f6-47fd-b510-21a50d14490e service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:34.215799 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1472/2895] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:34.633585 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4d7f9-0ac5-455a-ad6e-f508bc68bbd7 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:34.635105 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-45d4d7f9-0ac5-455a-ad6e-f508bc68bbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:34.648893 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4d7f9-0ac5-455a-ad6e-f508bc68bbd7 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:34.653993 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:34.654664 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:34.654944 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:34.655223 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] PATH_INFO: `/v3/projects/5439871b946842c18cd47f5b9f1f9545` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:34.656421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5439871b946842c18cd47f5b9f1f9545)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:34.658473 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:34.790249 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5439871b946842c18cd47f5b9f1f9545', 'request_id': 'req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:34.797256 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5439871b946842c18cd47f5b9f1f9545', 'request_id': 'req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:34.888687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5439871b946842c18cd47f5b9f1f9545', 'request_id': 'req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:34.900406 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] The token cache is being invalidate because project 5439871b946842c18cd47f5b9f1f9545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:47:34.900700 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:34.901360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-abebdbb9-a0f6-43c3-a9a2-6359e7d0af01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:34.902922 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1424/2896] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:47:34 2022] DELETE /identity/v3/projects/5439871b946842c18cd47f5b9f1f9545 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:34.907966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadefa25-ab6b-4d02-ba58-1cf88b4bd6bb None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:34.910599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadefa25-ab6b-4d02-ba58-1cf88b4bd6bb None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:34.910599 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadefa25-ab6b-4d02-ba58-1cf88b4bd6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:34.929422 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-fadefa25-ab6b-4d02-ba58-1cf88b4bd6bb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:34.930685 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-fadefa25-ab6b-4d02-ba58-1cf88b4bd6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:34.999350 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1473/2897] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:35.009670 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef4aef-2e09-4e7c-ab22-50ea142f0532 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:35.009937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef4aef-2e09-4e7c-ab22-50ea142f0532 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:35.010201 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef4aef-2e09-4e7c-ab22-50ea142f0532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:35.031065 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-f5ef4aef-2e09-4e7c-ab22-50ea142f0532 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:35.032290 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-f5ef4aef-2e09-4e7c-ab22-50ea142f0532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:35.041277 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1425/2898] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:35.045883 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadefa25-ab6b-4d02-ba58-1cf88b4bd6bb None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:35.057322 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadefa25-ab6b-4d02-ba58-1cf88b4bd6bb None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:35.062018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb5bd2-a199-4209-a8b1-dbfe8886200c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:35.062661 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb5bd2-a199-4209-a8b1-dbfe8886200c admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:35.062946 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb5bd2-a199-4209-a8b1-dbfe8886200c admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:35.063225 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb5bd2-a199-4209-a8b1-dbfe8886200c admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:35.063877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb5bd2-a199-4209-a8b1-dbfe8886200c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:47:35.064182 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb5bd2-a199-4209-a8b1-dbfe8886200c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:35.066707 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb5bd2-a199-4209-a8b1-dbfe8886200c admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:35.068384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:35.079373 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1474/2899] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:47:35.085549 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef4aef-2e09-4e7c-ab22-50ea142f0532 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:35.095282 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef4aef-2e09-4e7c-ab22-50ea142f0532 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:35.100261 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c791a7f-1b9f-4b37-b177-43d70ead5f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:35.100915 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c791a7f-1b9f-4b37-b177-43d70ead5f5d admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:35.101156 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c791a7f-1b9f-4b37-b177-43d70ead5f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:35.101443 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c791a7f-1b9f-4b37-b177-43d70ead5f5d admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:35.103220 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c791a7f-1b9f-4b37-b177-43d70ead5f5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:35.104039 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c791a7f-1b9f-4b37-b177-43d70ead5f5d admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:35.241038 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1426/2900] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:47:35 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:47:35.247393 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb5bd2-a199-4209-a8b1-dbfe8886200c admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:35.258469 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb5bd2-a199-4209-a8b1-dbfe8886200c admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:35.322853 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879aa83-e8a2-4156-b427-6fb8713f0de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:35.323384 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879aa83-e8a2-4156-b427-6fb8713f0de6 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:35.323660 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879aa83-e8a2-4156-b427-6fb8713f0de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:35.323931 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879aa83-e8a2-4156-b427-6fb8713f0de6 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:35.324554 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879aa83-e8a2-4156-b427-6fb8713f0de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:35.326528 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879aa83-e8a2-4156-b427-6fb8713f0de6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:35.414877 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1475/2901] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:47:35 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:47:35.418890 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c791a7f-1b9f-4b37-b177-43d70ead5f5d admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:35.427074 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:35.431247 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c791a7f-1b9f-4b37-b177-43d70ead5f5d admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:35.435939 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d71d1-33e5-4874-95c9-ae9f74e993bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:35.436543 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d71d1-33e5-4874-95c9-ae9f74e993bd admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:35.436912 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d71d1-33e5-4874-95c9-ae9f74e993bd admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:35.437183 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d71d1-33e5-4874-95c9-ae9f74e993bd admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:35.437815 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d71d1-33e5-4874-95c9-ae9f74e993bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:35.439453 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d71d1-33e5-4874-95c9-ae9f74e993bd admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:35.450684 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1427/2902] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:35.454994 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879aa83-e8a2-4156-b427-6fb8713f0de6 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:35.465394 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879aa83-e8a2-4156-b427-6fb8713f0de6 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:35.470107 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1080a1-4d7a-49d4-a04f-48df3be6a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:35.470747 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1080a1-4d7a-49d4-a04f-48df3be6a590 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:35.471029 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1080a1-4d7a-49d4-a04f-48df3be6a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:35.471298 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1080a1-4d7a-49d4-a04f-48df3be6a590 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:35.472635 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1080a1-4d7a-49d4-a04f-48df3be6a590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:35.474285 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1080a1-4d7a-49d4-a04f-48df3be6a590 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:35.485603 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1476/2903] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:35.490926 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d71d1-33e5-4874-95c9-ae9f74e993bd admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:35.501571 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d71d1-33e5-4874-95c9-ae9f74e993bd admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:35.506405 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ae241-ed01-41b3-97c2-d0986b4ecbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:35.507101 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ae241-ed01-41b3-97c2-d0986b4ecbd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:35.507382 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ae241-ed01-41b3-97c2-d0986b4ecbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:35.507690 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ae241-ed01-41b3-97c2-d0986b4ecbd7 admin admin] PATH_INFO: `/v3/projects/bc12d293bc594061919c45ef0552443a/users/558a99871baa44dda9d472a28af4f10b/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:35.524580 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ae241-ed01-41b3-97c2-d0986b4ecbd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc12d293bc594061919c45ef0552443a, user_id=558a99871baa44dda9d472a28af4f10b, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:35.527420 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ae241-ed01-41b3-97c2-d0986b4ecbd7 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:35.602985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1428/2904] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:47:35 2022] PUT /identity/v3/projects/bc12d293bc594061919c45ef0552443a/users/558a99871baa44dda9d472a28af4f10b/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:35.614460 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823eead-3d7d-4252-918a-44be76e4840e None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:35.614786 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823eead-3d7d-4252-918a-44be76e4840e None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:35.615064 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823eead-3d7d-4252-918a-44be76e4840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:35.644970 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-0823eead-3d7d-4252-918a-44be76e4840e None None] MFA Rules not processed for user `558a99871baa44dda9d472a28af4f10b`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:35.646220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-0823eead-3d7d-4252-918a-44be76e4840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:35.713434 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1477/2905] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:35.726093 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ae241-ed01-41b3-97c2-d0986b4ecbd7 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:35.727834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-dd4ae241-ed01-41b3-97c2-d0986b4ecbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:35.786161 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ae241-ed01-41b3-97c2-d0986b4ecbd7 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:35.789853 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f280e5-da3b-415f-a08d-17ac12b6d2fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:35.790459 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f280e5-da3b-415f-a08d-17ac12b6d2fb service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:35.790766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f280e5-da3b-415f-a08d-17ac12b6d2fb service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:35.791040 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f280e5-da3b-415f-a08d-17ac12b6d2fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:35.799394 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f280e5-da3b-415f-a08d-17ac12b6d2fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:35.801692 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f280e5-da3b-415f-a08d-17ac12b6d2fb service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:35.810767 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1429/2906] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:45.710216 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:45.806556 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:47.115670 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823eead-3d7d-4252-918a-44be76e4840e None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:47.181399 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823eead-3d7d-4252-918a-44be76e4840e None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:47.186243 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:47.186913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:47.187196 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:47.187470 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] PATH_INFO: `/v3/users/558a99871baa44dda9d472a28af4f10b` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:47.188594 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=558a99871baa44dda9d472a28af4f10b)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:47.190561 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:47.349593 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '558a99871baa44dda9d472a28af4f10b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:47.350220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '558a99871baa44dda9d472a28af4f10b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:47.350881 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '558a99871baa44dda9d472a28af4f10b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:47.436058 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '558a99871baa44dda9d472a28af4f10b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:47.446117 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '558a99871baa44dda9d472a28af4f10b'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:47.466408 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1478/2907] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:47:47 2022] DELETE /identity/v3/users/558a99871baa44dda9d472a28af4f10b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:47.471293 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32222d-8bee-4719-86d8-49c9ab3fbfb1 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:47.471878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32222d-8bee-4719-86d8-49c9ab3fbfb1 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:47.471878 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32222d-8bee-4719-86d8-49c9ab3fbfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:47.493703 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-3d32222d-8bee-4719-86d8-49c9ab3fbfb1 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:47.495029 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3d32222d-8bee-4719-86d8-49c9ab3fbfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:47.559592 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1430/2908] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:47.571482 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:47.573050 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:47.633532 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df66e9-aee2-46a1-90f6-a88f2f0e2b6e admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:47.637178 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5427a-adca-47a1-a354-1a2f11e2d9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:47.637789 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5427a-adca-47a1-a354-1a2f11e2d9bc service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:47.638073 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5427a-adca-47a1-a354-1a2f11e2d9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:47.638342 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5427a-adca-47a1-a354-1a2f11e2d9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:47.648173 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5427a-adca-47a1-a354-1a2f11e2d9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:47.649839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5427a-adca-47a1-a354-1a2f11e2d9bc service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:47.659736 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1479/2909] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:48.204431 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32222d-8bee-4719-86d8-49c9ab3fbfb1 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:48.208660 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-3d32222d-8bee-4719-86d8-49c9ab3fbfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:48.221037 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32222d-8bee-4719-86d8-49c9ab3fbfb1 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:48.227083 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:48.227737 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:48.228014 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:48.228312 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] PATH_INFO: `/v3/projects/bc12d293bc594061919c45ef0552443a` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:48.229525 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc12d293bc594061919c45ef0552443a)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:48.243158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:48.381985 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc12d293bc594061919c45ef0552443a', 'request_id': 'req-443f93d9-587c-4d93-9a1a-9c23b7804abb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:48.389158 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc12d293bc594061919c45ef0552443a', 'request_id': 'req-443f93d9-587c-4d93-9a1a-9c23b7804abb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:48.475940 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc12d293bc594061919c45ef0552443a', 'request_id': 'req-443f93d9-587c-4d93-9a1a-9c23b7804abb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:48.487836 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] The token cache is being invalidate because project bc12d293bc594061919c45ef0552443a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:47:48.488135 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:48.488833 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-443f93d9-587c-4d93-9a1a-9c23b7804abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:47:48.494735 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1431/2910] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:47:48 2022] DELETE /identity/v3/projects/bc12d293bc594061919c45ef0552443a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:48.495286 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a2f50-52d3-464b-ae31-e64322301eeb None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:48.495757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a2f50-52d3-464b-ae31-e64322301eeb None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:48.495884 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a2f50-52d3-464b-ae31-e64322301eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:48.515279 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-9b6a2f50-52d3-464b-ae31-e64322301eeb None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:48.516506 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-9b6a2f50-52d3-464b-ae31-e64322301eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:48.535819 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:48.577548 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1480/2911] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:48.588322 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d75b6-f8f1-479e-a5d4-4c06137b9113 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:48.588601 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d75b6-f8f1-479e-a5d4-4c06137b9113 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:48.588867 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d75b6-f8f1-479e-a5d4-4c06137b9113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:48.609834 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-1e1d75b6-f8f1-479e-a5d4-4c06137b9113 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:48.611119 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-1e1d75b6-f8f1-479e-a5d4-4c06137b9113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:48.619850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1432/2912] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:48.625120 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a2f50-52d3-464b-ae31-e64322301eeb None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:48.636544 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a2f50-52d3-464b-ae31-e64322301eeb None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:48.641273 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c74c24-5821-4532-a8a7-d749592c90cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:48.641966 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c74c24-5821-4532-a8a7-d749592c90cd admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:48.642530 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c74c24-5821-4532-a8a7-d749592c90cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:48.642807 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c74c24-5821-4532-a8a7-d749592c90cd admin admin] PATH_INFO: `/v3/domains` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:48.643457 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c74c24-5821-4532-a8a7-d749592c90cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 17:47:48.643794 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c74c24-5821-4532-a8a7-d749592c90cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:48.646061 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c74c24-5821-4532-a8a7-d749592c90cd admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:48.658717 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1481/2913] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 22 17:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 17:47:48.663377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d75b6-f8f1-479e-a5d4-4c06137b9113 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:48.672983 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d75b6-f8f1-479e-a5d4-4c06137b9113 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:48.677961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f241bf-657e-45b1-adbd-19b266c25273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:48.678616 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f241bf-657e-45b1-adbd-19b266c25273 admin admin] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:48.678894 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f241bf-657e-45b1-adbd-19b266c25273 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:48.679164 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f241bf-657e-45b1-adbd-19b266c25273 admin admin] PATH_INFO: `/v3/projects` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:48.679792 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f241bf-657e-45b1-adbd-19b266c25273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:48.681719 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f241bf-657e-45b1-adbd-19b266c25273 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:48.816498 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1433/2914] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 22 17:47:48 2022] POST /identity/v3/projects => generated 381 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:47:48.821418 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c74c24-5821-4532-a8a7-d749592c90cd admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:48.834284 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c74c24-5821-4532-a8a7-d749592c90cd admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:48.898445 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fe8a3-eda6-4439-9673-7af151b9f6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:48.899092 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fe8a3-eda6-4439-9673-7af151b9f6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:48.899364 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fe8a3-eda6-4439-9673-7af151b9f6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:48.899634 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fe8a3-eda6-4439-9673-7af151b9f6b6 admin admin] PATH_INFO: `/v3/users` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:48.900268 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790fe8a3-eda6-4439-9673-7af151b9f6b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:48.902203 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790fe8a3-eda6-4439-9673-7af151b9f6b6 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:49.043712 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1482/2915] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 22 17:47:48 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 17:47:49.047859 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f241bf-657e-45b1-adbd-19b266c25273 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:49.059474 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f241bf-657e-45b1-adbd-19b266c25273 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:49.064235 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c119f26-9a3a-4160-ad92-44bcfe79d3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:49.064850 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c119f26-9a3a-4160-ad92-44bcfe79d3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:49.065128 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c119f26-9a3a-4160-ad92-44bcfe79d3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:49.065421 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c119f26-9a3a-4160-ad92-44bcfe79d3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:49.067193 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c119f26-9a3a-4160-ad92-44bcfe79d3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:49.068756 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c119f26-9a3a-4160-ad92-44bcfe79d3b3 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:49.079295 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1434/2916] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:49.084263 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fe8a3-eda6-4439-9673-7af151b9f6b6 admin admin] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:49.094757 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fe8a3-eda6-4439-9673-7af151b9f6b6 admin admin] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:49.099891 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc266e1e-6abb-46a6-a40f-d48a48f40217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:49.100528 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266e1e-6abb-46a6-a40f-d48a48f40217 admin admin] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:49.100834 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266e1e-6abb-46a6-a40f-d48a48f40217 admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:49.101110 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266e1e-6abb-46a6-a40f-d48a48f40217 admin admin] PATH_INFO: `/v3/roles` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:49.101754 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc266e1e-6abb-46a6-a40f-d48a48f40217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:49.103403 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc266e1e-6abb-46a6-a40f-d48a48f40217 admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:49.115586 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1483/2917] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 22 17:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 17:47:49.119774 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c119f26-9a3a-4160-ad92-44bcfe79d3b3 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:49.130765 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c119f26-9a3a-4160-ad92-44bcfe79d3b3 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:49.135423 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde304c5-d311-453e-ab55-043eedd05833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:49.136073 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde304c5-d311-453e-ab55-043eedd05833 admin admin] REQUEST_METHOD: `PUT` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:49.136352 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde304c5-d311-453e-ab55-043eedd05833 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:49.136627 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde304c5-d311-453e-ab55-043eedd05833 admin admin] PATH_INFO: `/v3/projects/e14445197d18423bbd90a1c6211451dd/users/decf7bb66eec469499237b4ebcf9b4eb/roles/13accf89c12040508974db47c6905b31` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:49.151541 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde304c5-d311-453e-ab55-043eedd05833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14445197d18423bbd90a1c6211451dd, user_id=decf7bb66eec469499237b4ebcf9b4eb, role_id=13accf89c12040508974db47c6905b31)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:49.154218 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde304c5-d311-453e-ab55-043eedd05833 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:49.231560 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1435/2918] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 22 17:47:49 2022] PUT /identity/v3/projects/e14445197d18423bbd90a1c6211451dd/users/decf7bb66eec469499237b4ebcf9b4eb/roles/13accf89c12040508974db47c6905b31 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:47:49.236863 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2469adb-d19a-47a2-82c2-5a8e5ec4a2d2 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:49.237148 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2469adb-d19a-47a2-82c2-5a8e5ec4a2d2 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:49.237420 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2469adb-d19a-47a2-82c2-5a8e5ec4a2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:49.258727 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-d2469adb-d19a-47a2-82c2-5a8e5ec4a2d2 None None] MFA Rules not processed for user `b8814a79ac534e6e8b81d70404045912`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:49.259925 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-d2469adb-d19a-47a2-82c2-5a8e5ec4a2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:49.322547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1484/2919] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:49.332988 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde304c5-d311-453e-ab55-043eedd05833 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:49.334780 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-dde304c5-d311-453e-ab55-043eedd05833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:49.357262 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:49.394469 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde304c5-d311-453e-ab55-043eedd05833 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:49.398264 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46120649-6d74-4cca-bf69-ae8b26fbfb5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:49.398904 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46120649-6d74-4cca-bf69-ae8b26fbfb5b service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:49.399184 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46120649-6d74-4cca-bf69-ae8b26fbfb5b service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:49.399458 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46120649-6d74-4cca-bf69-ae8b26fbfb5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:49.408801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46120649-6d74-4cca-bf69-ae8b26fbfb5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:49.410917 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46120649-6d74-4cca-bf69-ae8b26fbfb5b service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:49.420801 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1436/2920] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:47:50.594936 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4f306-995a-47d2-ab21-00c409f68141 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:50.595646 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4f306-995a-47d2-ab21-00c409f68141 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:50.596094 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4f306-995a-47d2-ab21-00c409f68141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:50.632967 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-56e4f306-995a-47d2-ab21-00c409f68141 None None] MFA Rules not processed for user `decf7bb66eec469499237b4ebcf9b4eb`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:50.634204 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-56e4f306-995a-47d2-ab21-00c409f68141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:50.652984 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:47:50.711056 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1485/2921] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:50.721687 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46120649-6d74-4cca-bf69-ae8b26fbfb5b service neutron] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:50.723820 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-46120649-6d74-4cca-bf69-ae8b26fbfb5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:50.783327 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46120649-6d74-4cca-bf69-ae8b26fbfb5b service neutron] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:50.787694 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bf62e-c87b-4cfb-ba6e-8fbe457406b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68088e59a2c64ff2948e0549f1fc7cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:50.788294 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bf62e-c87b-4cfb-ba6e-8fbe457406b4 service cinder] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:50.788572 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bf62e-c87b-4cfb-ba6e-8fbe457406b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:50.788846 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bf62e-c87b-4cfb-ba6e-8fbe457406b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:50.797444 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bf62e-c87b-4cfb-ba6e-8fbe457406b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:50.798941 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bf62e-c87b-4cfb-ba6e-8fbe457406b4 service cinder] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:50.807914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1437/2922] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:52.591474 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802a66-e61f-483d-9f1b-69b1fed63d49 None None] REQUEST_METHOD: `POST` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:52.592166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802a66-e61f-483d-9f1b-69b1fed63d49 None None] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:52.592166 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802a66-e61f-483d-9f1b-69b1fed63d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:52.612158 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.auth.core [None req-e7802a66-e61f-483d-9f1b-69b1fed63d49 None None] MFA Rules not processed for user `decf7bb66eec469499237b4ebcf9b4eb`. Rule list: `[]` (Enabled: `True`). {{(pid=82146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:47:52.614151 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e7802a66-e61f-483d-9f1b-69b1fed63d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:52.621119 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1486/2923] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:47:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:47:52.859677 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bf62e-c87b-4cfb-ba6e-8fbe457406b4 service cinder] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:52.869880 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bf62e-c87b-4cfb-ba6e-8fbe457406b4 service cinder] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:52.873492 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fe4de-e207-407d-960f-4e799f4bec6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:52.874086 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fe4de-e207-407d-960f-4e799f4bec6f service neutron] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:52.874360 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fe4de-e207-407d-960f-4e799f4bec6f service neutron] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:52.874664 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fe4de-e207-407d-960f-4e799f4bec6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:52.876381 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-d86fe4de-e207-407d-960f-4e799f4bec6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:52.886739 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fe4de-e207-407d-960f-4e799f4bec6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:52.888116 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fe4de-e207-407d-960f-4e799f4bec6f service neutron] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:52.896300 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1438/2924] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:47:56.969220 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7802a66-e61f-483d-9f1b-69b1fed63d49 None None] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:47:56.970733 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-e7802a66-e61f-483d-9f1b-69b1fed63d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:56.983481 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7802a66-e61f-483d-9f1b-69b1fed63d49 None None] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:47:56.987431 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1622d-ff61-40ce-8e5f-17389d73f8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c29a91e099ba4cd5886fd910d978fb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:47:56.988022 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1622d-ff61-40ce-8e5f-17389d73f8af service neutron] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:47:56.988566 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1622d-ff61-40ce-8e5f-17389d73f8af service neutron] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:47:56.989259 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1622d-ff61-40ce-8e5f-17389d73f8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:47:56.991103 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-72e1622d-ff61-40ce-8e5f-17389d73f8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:47:57.000839 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1622d-ff61-40ce-8e5f-17389d73f8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:47:57.002257 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1622d-ff61-40ce-8e5f-17389d73f8af service neutron] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:47:57.011815 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1487/2925] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:47:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:48:02.892782 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=49 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:48:07.007913 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=28 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:49:17.397491 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147de9a-95ec-4650-915d-a92ac7bdedf6 None None] REQUEST_METHOD: `POST` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:49:17.398180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147de9a-95ec-4650-915d-a92ac7bdedf6 None None] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:49:17.398180 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147de9a-95ec-4650-915d-a92ac7bdedf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:49:17.417509 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.auth.core [None req-0147de9a-95ec-4650-915d-a92ac7bdedf6 None None] MFA Rules not processed for user `decf7bb66eec469499237b4ebcf9b4eb`. Rule list: `[]` (Enabled: `True`). {{(pid=82145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 17:49:17.418766 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0147de9a-95ec-4650-915d-a92ac7bdedf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:49:17.425674 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1439/2926] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 22 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 17:49:27.412113 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=3 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:49:46.133975 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1622d-ff61-40ce-8e5f-17389d73f8af service neutron] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:49:46.135557 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-72e1622d-ff61-40ce-8e5f-17389d73f8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:49:46.207162 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1622d-ff61-40ce-8e5f-17389d73f8af service neutron] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:49:46.213351 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbc44b-d9c1-453a-8a89-11054190f325 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:49:46.213973 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc44b-d9c1-453a-8a89-11054190f325 service placement] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:49:46.214256 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc44b-d9c1-453a-8a89-11054190f325 service placement] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:49:46.214562 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc44b-d9c1-453a-8a89-11054190f325 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:49:46.216333 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-5dfbc44b-d9c1-453a-8a89-11054190f325 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:49:46.229598 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbc44b-d9c1-453a-8a89-11054190f325 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:49:46.231326 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbc44b-d9c1-453a-8a89-11054190f325 service placement] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:49:46.242349 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1488/2927] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:49:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:49:56.237592 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=32 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:50:00.281061 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147de9a-95ec-4650-915d-a92ac7bdedf6 None None] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:50:00.283701 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-0147de9a-95ec-4650-915d-a92ac7bdedf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:50:00.401522 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147de9a-95ec-4650-915d-a92ac7bdedf6 None None] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:50:00.412015 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633207a-5acd-44c0-a99a-7e9a86901874 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9628118c1df64fa4a7ea8ce37549d8b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:50:00.412937 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633207a-5acd-44c0-a99a-7e9a86901874 service nova] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:50:00.413356 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633207a-5acd-44c0-a99a-7e9a86901874 service nova] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:50:00.413750 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633207a-5acd-44c0-a99a-7e9a86901874 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:50:00.416513 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-8633207a-5acd-44c0-a99a-7e9a86901874 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:50:00.439576 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633207a-5acd-44c0-a99a-7e9a86901874 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:50:00.441573 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633207a-5acd-44c0-a99a-7e9a86901874 service nova] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:50:00.455108 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1440/2928] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:50:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 17:50:10.448916 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=34 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:50:32.028887 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbc44b-d9c1-453a-8a89-11054190f325 service placement] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:50:32.054836 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbc44b-d9c1-453a-8a89-11054190f325 service placement] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:50:32.060619 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632ff03-fd51-4394-9a8b-6f5b18238e44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:50:32.061265 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632ff03-fd51-4394-9a8b-6f5b18238e44 service placement] REQUEST_METHOD: `GET` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:50:32.061564 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632ff03-fd51-4394-9a8b-6f5b18238e44 service placement] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:50:32.061849 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632ff03-fd51-4394-9a8b-6f5b18238e44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:50:32.063677 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.fernet_utils [None req-6632ff03-fd51-4394-9a8b-6f5b18238e44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:50:32.077308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632ff03-fd51-4394-9a8b-6f5b18238e44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:50:32.079003 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632ff03-fd51-4394-9a8b-6f5b18238e44 service placement] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:50:32.092898 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1489/2929] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:50:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:50:32.166106 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633207a-5acd-44c0-a99a-7e9a86901874 service nova] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:50:32.177295 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633207a-5acd-44c0-a99a-7e9a86901874 service nova] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:50:32.190457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:50:32.191457 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:50:32.191961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:50:32.191961 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] PATH_INFO: `/v3/users/decf7bb66eec469499237b4ebcf9b4eb` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:50:32.193214 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=decf7bb66eec469499237b4ebcf9b4eb)` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:50:32.196377 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:50:32.397823 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'decf7bb66eec469499237b4ebcf9b4eb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:50:32.398675 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'decf7bb66eec469499237b4ebcf9b4eb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:50:32.399818 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'decf7bb66eec469499237b4ebcf9b4eb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:50:32.407842 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'decf7bb66eec469499237b4ebcf9b4eb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:50:32.422741 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.notifications [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'decf7bb66eec469499237b4ebcf9b4eb'} {{(pid=82145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:50:32.502710 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1441/2930] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 22 17:50:32 2022] DELETE /identity/v3/users/decf7bb66eec469499237b4ebcf9b4eb => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 17:50:32.789018 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632ff03-fd51-4394-9a8b-6f5b18238e44 service placement] Authenticating user token {{(pid=82146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:50:32.807175 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632ff03-fd51-4394-9a8b-6f5b18238e44 service placement] Validating token access rules against request {{(pid=82146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:50:32.878752 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8814a79ac534e6e8b81d70404045912', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14e122ee73be44deb3dfe27923205d92', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:50:32.879308 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:50:32.879587 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:50:32.879857 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] PATH_INFO: `/v3/projects/e14445197d18423bbd90a1c6211451dd` {{(pid=82146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:50:32.880998 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e14445197d18423bbd90a1c6211451dd)` {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:50:32.883121 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] RBAC: Authorization granted {{(pid=82146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:50:32.986756 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e14445197d18423bbd90a1c6211451dd', 'request_id': 'req-0fb0d064-13cf-4771-baca-403792503d7e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:50:32.993020 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e14445197d18423bbd90a1c6211451dd', 'request_id': 'req-0fb0d064-13cf-4771-baca-403792503d7e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:50:32.998783 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e14445197d18423bbd90a1c6211451dd', 'request_id': 'req-0fb0d064-13cf-4771-baca-403792503d7e'} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:50:33.110742 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] The token cache is being invalidate because project e14445197d18423bbd90a1c6211451dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 17:50:33.111547 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:50:33.111700 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG keystone.notifications [None req-0fb0d064-13cf-4771-baca-403792503d7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 17:50:33.113365 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: [pid: 82146|app: 0|req: 1490/2931] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 22 17:50:32 2022] DELETE /identity/v3/projects/e14445197d18423bbd90a1c6211451dd => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 17:50:42.498611 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:50:43.107471 dv-cinder-853941-2-iscsi devstack@keystone.service[82146]: DEBUG dbcounter [-] [82146] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 17:55:46.813297 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] Authenticating user token {{(pid=82145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 17:55:46.814810 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:55:46.961483 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c6d87-c68d-42ad-8575-30fe1ae1b929 admin admin] Validating token access rules against request {{(pid=82145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 17:55:47.021466 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f9a84-e257-42d4-a901-2a85fc6919e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b5021285c44db9b05f9d15ceaf797f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a99c33cfab44e22b69495fc410fa077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 17:55:47.022136 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f9a84-e257-42d4-a901-2a85fc6919e9 service placement] REQUEST_METHOD: `GET` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 17:55:47.022426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f9a84-e257-42d4-a901-2a85fc6919e9 service placement] SCRIPT_NAME: `/identity` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 17:55:47.022653 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f9a84-e257-42d4-a901-2a85fc6919e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 17:55:47.024426 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.fernet_utils [None req-a44f9a84-e257-42d4-a901-2a85fc6919e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 17:55:47.036914 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f9a84-e257-42d4-a901-2a85fc6919e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 17:55:47.038545 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f9a84-e257-42d4-a901-2a85fc6919e9 service placement] RBAC: Authorization granted {{(pid=82145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 17:55:47.048920 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: [pid: 82145|app: 0|req: 1442/2932] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 22 17:55:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 17:55:57.044398 dv-cinder-853941-2-iscsi devstack@keystone.service[82145]: DEBUG dbcounter [-] [82145] Writing DB stats keystone:SELECT=89 {{(pid=82145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}